An employee is paid an hourly rate for however many hours he wishes to work. What will happen to the number of hours per week he works if the rate is increased?

It depends. The substitution effect says work more, the income effect says work less. The result depends on which one dominates.

If I use my rational model to give instructions to someone who blindly follows, lacking his own model, and he achieves his goal, I would not say he succeeded arationally. The fact that the agent did not contain the rationality but only received the conclusions does not mean the rationality was not important to the process.

If I extend the situation, so that I use my rational model to describe simple rules that produce success in a broad class of situations, and an agent lacking a model succeed by following my rules, this still is not arational. The agent is still using my rationality indirectly.

If I take the simple rules I derived from my rational model, and design a device to mechanically follow the rules, the device will work, not arationally, but indirectly rationally. This is your control system. It does not contain a model, it just blindly follows rules that were produced by rationality.

It sounds like you think that, if someone thinks about control theory the way I do, they will make prediction mistakes or bad decisions. And you want to keep people from making prediction mistakes or bad decisions, but first you have to make them see that there's something wrong with their thinking, and you don't know a direct argument that will make them see that, so you have to use a lot of arguments from examples. Can you say more directly what kinds of prediction mistakes or bad decisions you think that people who think about control theory the way I do will make?

The cruise control does not sense the gradient of the road, nor the head wind. It senses the speed of the car. It may be tuned for some broad characteristics of the vehicle, but it does not itself know those characteristics, or sense when they change, such as when passengers get in and out.

I didn't expect that the cruise control would be able to do that (without another controller for its tuning). That's why one of my list of sufficient conditions for optimality of a PID controller was, "the system is a second-order linear system with constant coefficients". If the coefficients change, then the same PID controller may not be optimal. Did you expect that I would have expected the cruise control to be able to sense changes in the characteristics of the vehicle? Or are you trying to say that, if someone thought about control systems the way I did, they would have expected the cruise control to be able to know when the vehicle characteristics change, except if they were thinking carefully at the time the way I was? For example, are you trying to say that I might have had this mistaken expectation if I was only thinking about the cruise control as part of thinking about something else? And you want to make me see that there's something wrong with my thinking that makes me make prediction mistakes when I'm not thinking carefully?

An implicit model is one in which functional relationships are expressed not as explicit functions y=f(x), but as relations g(x,y)=k.

It sounds like you want to say that a controller doesn't have any implicit model unless it has a separate, identifiable physical part or software data structure that expresses a relationship and has no other function. If one controller is mathematically equivalent to another controller that does have a separate, identifiable part that expresses a relationship, but the controller itself doesn't have a separate, identifiable part that expresses a relationship, does it still not have an implicit model?

Linear controllers are optimal for many control problems that are natural limiting cases or approximations of real-world families of control problems. In a control problem where a linear controller is the Bayes-optimal controller, it is literally impossible for any controller with different outputs from the linear controller to have a lower average cost. Even if a controller was made of separate identifiable parts that implemented the separate identifiable parts of Bayesian sequential decision theory, and even if some of those parts expressed relations between past, present, or future perception signals, reference signals, control signals, or system states, the controller still couldn't do any better than the optimal linear controller. And all the information that a Bayesian optimal controller could use is already in either the state of the optimal linear controller or the state of the system which the controller has been controlling. If a Bayesian decision-theoretic controller had to take over from an optimal linear controller, it would have no use for any more information than the state of the controller and the perception and reference signals at the controller, which is also the only information that the linear controller was able to use. If the Bayesian controller was given more information about past reference signals or perception signals, that would not help its performance. At any time, the posterior belief distribution in a Bayesian controller can be set equal to a new posterior belief distribution defined using only the perception signal, the reference signal, and the state that the optimal linear controller would have had at that time, and the Bayesian controller will still have optimal performance. And the state in an optimal linear controller can be set equal to a new state defined using only the perception signal, the reference signal, and the posterior belief distribution that a Bayesian controller would have had at that time. This means that, whatever information processing a Bayesian optimal controller for a linear-quadratic-Gaussian control problem would be doing that would affect the control signal, the optimal linear controller (together with the system it is controlling) is already doing that information processing. They are mathematically equivalent.

A linear controller can be optimal for more than one control problem. To define a linear controller's implicit model of the system and disturbances, you need a model of the reference signal and the cost functional; to define a linear controller's implicit model of the reference signal, you need a model of the cost functional and the system and disturbances; and to define a linear controller's implicit model of the cost functional, you need a model of the reference signal and of the system and disturbances. Some of these implicit models are only defined up to a constant factor. And a linear controller that is optimal for some control problems can also perform well on other control problems that are near them.

(An optimal Bayesian controller for a linear-quadratic-Gaussian control problem isn't able to change its model when the system coefficients or the statistical properties of the disturbances or reference signal change. This is because the controller would have no prior belief that a change was possible. All of the controller's prior probability would be on the belief that the system and disturbances and reference signal would act like the problem the controller was designed to be optimal for. If the controller had any prior probability on any other belief, it would make decisions that wouldn't be optimal for the problem it was designed to be optimal for.)

Now, I am not explaining control systems merely to explain control systems. The relevance to rationality is that they funnel reality into a narrow path in configuration space by entirely arational means, and thus constitute a proof by example that this is possible.

It sounds like you are saying that the math for when a controller works doesn't leave any shadow at all in the math of what a good controller does. If you aren't saying that, then I disagree less with what you have said.

This must raise the question, how much of the neural functioning of a living organism, human or lesser, operates by similar means?

Agreed.

\5. What relates questions 3 and 4 to the subject of this article?

Are questions 3 and 4 situations in which people who think about control theory the way I do might make prediction mistakes when they aren't thinking carefully? Are they situations in which the employer has a mistaken implicit model of how to increase (reference signal) the employee's hours (perception signal) by changing his wages (control signal) and the medical bureaucrats have a mistaken implicit model of how to decrease (reference signal) the doctor's time per patient (perception signal) by controlling his target (control signal)? Are they situations in which the employer has a mistaken belief about the control system inside the employee and the medical bureaucrats have a mistaken belief about the control system inside the doctor?

\6. Controller: o = c×(r-p). Environment: dp/dt = k×o + d. o, r, and p as above; c and k are constants; d is an arbitrary function of time (the disturbance). How fast and how accurately does this controller reject the disturbance and track the reference?

Errors will decay exponentially with rate constant k×c, if k×c is positive.

If d is constant and r is constant, then p = r+d/(k×c).

If d is zero and r = m×t, then p = r-m/(k×c): p will lag by m/(k×c). The P controller implicitly predicts that the future changes of r will on average be equal to the integral of d, which is zero. Because the average future change of r is something other than zero, on average the P controller lags. A tuned PI controller could implicitly learn m and implicitly predict future changes in r and not lag on average.

If the controller had the control law o = -c×p, its implicit model would be that r(t) will on average be equal to the integral of e^(-k×c×s)×d(t-s) with respect to s for s from zero to infinity, and that there is no information about future values of r in the current value of r that's not also in the current value of that integral. If the controller had the implicit model that r was constant at zero, its control law would be o = -∞×p, because in our model of the environment that we are using to define the controller's implicit model, there are no measurement errors, no delayed effects, and no control costs.

Exercise 4. When targets were imposed on British GPs, the effect at my practice was that only a few appointments were available. I had to sit by the phone from 8.30am when the practice opened, phoning, getting the engaged tone, repeatedly pressing the redial button. Then I got the appointment time I wanted. Phone later in the day and there were no appointments available.

A GP may decide that his amour propre (signalling?) is more important than conforming to targets.

Experience on targets appears to indicate that people find ways of meeting the target, which may or may not be by achieving what the target-setter wished to achieve.

"Now, I am not explaining control systems merely to explain control systems. The relevance to rationality is that they funnel reality into a narrow path in configuration space by entirely arational means, and thus constitute a proof by example that this is possible."

I don't think you needed control systems to show this. Gravity itself is as much of a 'control system' - it minimizes the potential energy of the system! Heck, if you're doing that, lots of laws of physics fit that definition - they narrow down the set of possible realities...

" This must raise the question, how much of the neural functioning of a living organism, human or lesser, operates by similar means? "

So, I'm still not sure what you mean by 'similar means'.

We know the broad overview of how brains work - sensory neurons get triggered, they trigger other neurons, and through some web of complex things motor neurons eventually get triggered to give outputs. The stuff in the middle is hard; some of it can be described as "memory" (patterns that somehow represent past inputs), some can be represented various other abstractions. Control systems are probably good ways of interpreting a lot of combinations of neurons, and some have been brought up here. It seems unlikely that they would capture all of them - but if you stretch the analogy enough, perhaps it can.

"And how much of the functioning of an artificial organism must be designed to use these means? "

Must? I'd guess absolutely none. The way you have described them, control systems are not basic - for 'future perception' to truly determine current actions, would break causality. So it's certainly possible to describe/build an artificial organism without using control systems, though it seems like it would be pretty inconvenient and pointlessly harder than an already impossible problem given how useful you're describing them to be.

OK. So let'ts take a controller with an explicit (I hope you agree) model, the Smith predictor. The controller as a whole has a model, but the subsystem C(z) (in the wiki example) has not (in your terms).

Or better yet, a Model Reference Adaptive Control. The system as a whole IS predictive, uses models, etc.. but the "core" controller subsystem does "not".

Then I'd argue that in the simple PID case, the engineer does the job of the Model/Adjusting Mechanism, and it's a fundamental part of the implementation process (you don't just buy a PID and install it without tuning it first!)

So, in every control systems there is a model. It's only that when the plant is "simple enough" and invariant in the long term the model/adjusting subsystem is implemented in wetware, and only used during install.

This is just arguing semantics, though.

All control systems DO have models of what they are controlling. However, the models are typically VERY simple.

A good principle for constructing control systems are: Given that I have a very simple model, how do I optimize it?

The models I learned about in cybernetics were all linear, implemented as matrices, resistors and capacitors, or discrete time step filters. The most important thing was to show that the models and reality together did not result in amplification of oscillations. Then one made sure that the system actually did some controlling, and then one could fine tune it to reality to make it faster, more stable, etc.

One big advantage of linear models is that they can be inverted, and eigenvectors found. Doing equivalent stuff for other kinds of models is often very difficult, requiring lots of computation, or is simply impossible.

As has someone has written before here: It is mathematically justified to consider linear control systems as having statistical models of reality, typically involving gaussian distributions.

Kim Øyhus

Exercise 1. Assuming the truth of the statement that the candle has the effect of raising the temperature of the thermostat like that, (I do not have the knowledge to state whether that is or is not the case) the room temperature will oscillate between 15 and 16 degrees as the temperature of the thermostat continues to oscillate between 20 and 21, until the candle burns out.

I agree with the thesis; the referenced paper is really interesting, but the article on LW is a bit long-winded in trying to convey the notion that "there is no internal model". Amusingly the paper's title is "Internal models in the cerebellum"!

Great post! It clears up many of my misunderstandings and misconceptions I had after reading your last one.

I certainly have to agree, my concept of a implicit model is very problematic. And control systems are creepy!

Exercise 2. The accelerator will go to maximum, and the driver would have to brake maximally, until something burned out or the driver put the car out of gear or turned off the cruise control.