It sounds like you think that, if someone thinks about control theory the way I do, they will make prediction mistakes or bad decisions. And you want to keep people from making prediction mistakes or bad decisions, but first you have to make them see that there's something wrong with their thinking, and you don't know a direct argument that will make them see that, so you have to use a lot of arguments from examples. Can you say more directly what kinds of prediction mistakes or bad decisions you think that people who think about control theory the way I do will make?
The cruise control does not sense the gradient of the road, nor the head wind. It senses the speed of the car. It may be tuned for some broad characteristics of the vehicle, but it does not itself know those characteristics, or sense when they change, such as when passengers get in and out.
I didn't expect that the cruise control would be able to do that (without another controller for its tuning). That's why one of my list of sufficient conditions for optimality of a PID controller was, "the system is a second-order linear system with constant coefficients". If the coefficients change, then the same PID controller may not be optimal. Did you expect that I would have expected the cruise control to be able to sense changes in the characteristics of the vehicle? Or are you trying to say that, if someone thought about control systems the way I did, they would have expected the cruise control to be able to know when the vehicle characteristics change, except if they were thinking carefully at the time the way I was? For example, are you trying to say that I might have had this mistaken expectation if I was only thinking about the cruise control as part of thinking about something else? And you want to make me see that there's something wrong with my thinking that makes me make prediction mistakes when I'm not thinking carefully?
An implicit model is one in which functional relationships are expressed not as explicit functions y=f(x), but as relations g(x,y)=k.
It sounds like you want to say that a controller doesn't have any implicit model unless it has a separate, identifiable physical part or software data structure that expresses a relationship and has no other function. If one controller is mathematically equivalent to another controller that does have a separate, identifiable part that expresses a relationship, but the controller itself doesn't have a separate, identifiable part that expresses a relationship, does it still not have an implicit model?
Linear controllers are optimal for many control problems that are natural limiting cases or approximations of real-world families of control problems. In a control problem where a linear controller is the Bayes-optimal controller, it is literally impossible for any controller with different outputs from the linear controller to have a lower average cost. Even if a controller was made of separate identifiable parts that implemented the separate identifiable parts of Bayesian sequential decision theory, and even if some of those parts expressed relations between past, present, or future perception signals, reference signals, control signals, or system states, the controller still couldn't do any better than the optimal linear controller. And all the information that a Bayesian optimal controller could use is already in either the state of the optimal linear controller or the state of the system which the controller has been controlling. If a Bayesian decision-theoretic controller had to take over from an optimal linear controller, it would have no use for any more information than the state of the controller and the perception and reference signals at the controller, which is also the only information that the linear controller was able to use. If the Bayesian controller was given more information about past reference signals or perception signals, that would not help its performance. At any time, the posterior belief distribution in a Bayesian controller can be set equal to a new posterior belief distribution defined using only the perception signal, the reference signal, and the state that the optimal linear controller would have had at that time, and the Bayesian controller will still have optimal performance. And the state in an optimal linear controller can be set equal to a new state defined using only the perception signal, the reference signal, and the posterior belief distribution that a Bayesian controller would have had at that time. This means that, whatever information processing a Bayesian optimal controller for a linear-quadratic-Gaussian control problem would be doing that would affect the control signal, the optimal linear controller (together with the system it is controlling) is already doing that information processing. They are mathematically equivalent.
A linear controller can be optimal for more than one control problem. To define a linear controller's implicit model of the system and disturbances, you need a model of the reference signal and the cost functional; to define a linear controller's implicit model of the reference signal, you need a model of the cost functional and the system and disturbances; and to define a linear controller's implicit model of the cost functional, you need a model of the reference signal and of the system and disturbances. Some of these implicit models are only defined up to a constant factor. And a linear controller that is optimal for some control problems can also perform well on other control problems that are near them.
(An optimal Bayesian controller for a linear-quadratic-Gaussian control problem isn't able to change its model when the system coefficients or the statistical properties of the disturbances or reference signal change. This is because the controller would have no prior belief that a change was possible. All of the controller's prior probability would be on the belief that the system and disturbances and reference signal would act like the problem the controller was designed to be optimal for. If the controller had any prior probability on any other belief, it would make decisions that wouldn't be optimal for the problem it was designed to be optimal for.)
Now, I am not explaining control systems merely to explain control systems. The relevance to rationality is that they funnel reality into a narrow path in configuration space by entirely arational means, and thus constitute a proof by example that this is possible.
It sounds like you are saying that the math for when a controller works doesn't leave any shadow at all in the math of what a good controller does. If you aren't saying that, then I disagree less with what you have said.
This must raise the question, how much of the neural functioning of a living organism, human or lesser, operates by similar means?
Agreed.
\5. What relates questions 3 and 4 to the subject of this article?
Are questions 3 and 4 situations in which people who think about control theory the way I do might make prediction mistakes when they aren't thinking carefully? Are they situations in which the employer has a mistaken implicit model of how to increase (reference signal) the employee's hours (perception signal) by changing his wages (control signal) and the medical bureaucrats have a mistaken implicit model of how to decrease (reference signal) the doctor's time per patient (perception signal) by controlling his target (control signal)? Are they situations in which the employer has a mistaken belief about the control system inside the employee and the medical bureaucrats have a mistaken belief about the control system inside the doctor?
\6. Controller: o = c×(r-p). Environment: dp/dt = k×o + d. o, r, and p as above; c and k are constants; d is an arbitrary function of time (the disturbance). How fast and how accurately does this controller reject the disturbance and track the reference?
Errors will decay exponentially with rate constant k×c, if k×c is positive.
If d is constant and r is constant, then p = r+d/(k×c).
If d is zero and r = m×t, then p = r-m/(k×c): p will lag by m/(k×c). The P controller implicitly predicts that the future changes of r will on average be equal to the integral of d, which is zero. Because the average future change of r is something other than zero, on average the P controller lags. A tuned PI controller could implicitly learn m and implicitly predict future changes in r and not lag on average.
If the controller had the control law o = -c×p, its implicit model would be that r(t) will on average be equal to the integral of e^(-k×c×s)×d(t-s) with respect to s for s from zero to infinity, and that there is no information about future values of r in the current value of r that's not also in the current value of that integral. If the controller had the implicit model that r was constant at zero, its control law would be o = -∞×p, because in our model of the environment that we are using to define the controller's implicit model, there are no measurement errors, no delayed effects, and no control costs.
Followup to: What is control theory?
I mentioned in my post testing the water on this subject that control systems are not intuitive until one has learnt to understand them. The point I am going to talk about is one of those non-intuitive features of the subject. It is (a) basic to the very idea of a control system, and (b) something that almost everyone gets wrong when they first encounter control systems.
I'm going to address just this one point, not in order to ignore the rest, but because the discussion arising from my last post has shown that this is presently the most important thing.
There is a great temptation to think that to control a variable -- that is, to keep it at a desired value in spite of disturbing influences -- the controller must contain a model of the process to be controlled and use it to calculate what actions will have the desired effect. In addition, it must measure the disturbances or better still, predict them in advance and what effect they will have, and take those into account in deciding its actions.
In terms more familiar here, the temptation to think that to bring about desired effects in the world, one must have a model of the relevant parts of the world and predict what actions will produce the desired results.
However, this is absolutely wrong. This is not a minor mistake or a small misunderstanding; it is the pons asinorum of the subject.
Note the word "must". It is not disputed that one can use models and predictions, only that one must, that the task inherently requires it.
A control system can work without having any model of what it is controlling.
The designer will have a model. For the room thermostat, he must know that the heating should turn on when the room is too cold and off when it is too hot, rather than the other way around, and he must arrange that the source of heat is powerful enough. The controller he designs does not know that; it merely does that. (Compare the similar relationship between evolution and evolved organisms. How evolution works is not how the evolved organism works, nor is how a designer works how the designed system works.) For a cruise control, he must choose the parameters of the controller, taking into account the engine's response to the accelerator pedal. The resulting control system, however, contains no representation of that. According to the HowStuffWorks article, they typically use nothing more complicated than proportional or PID control. The parameters are chosen by the designer according to his knowledge about the system; the parameters themselves are not something the controller knows about the system.
It is possible to design control systems that do contain models, but it is not inherent to the task of control. This is what model-based controllers look like. (Thanks to Tom Talbot for that reference.) Pick up any book on model-based control to see more examples. There are signals within the control system that are designed to relate to each other in the same way as do corresponding properties of the world outside. That is what a model is. There is nothing even slightly resembling that in a thermostat or a cruise control. Nor is there in the knee-jerk tendon reflex. Whether there are models elsewhere in the human body is an empirical matter, to be decided by investigations such as those in the linked paper. To merely be entangled with the outside world is not what it is, to be a model.
Within the Alien Space Bat Prison Cell, the thermostat is flicking a switch one way when the needle is to the left of the mark, and the other when it is to the right. The cruise control is turning a knob by an amount proportional to the distance between the needle and the mark. Neither of them knows why. Neither of them knows what is outside the cell. Neither of them cares whether what they are doing is working. They just do it, and they work.
A control system can work without having any knowledge of the external disturbances.
The thermostat does not know that the sun is shining in through the window. It only knows the current temperature. The cruise control does not sense the gradient of the road, nor the head wind. It senses the speed of the car. It may be tuned for some broad characteristics of the vehicle, but it does not itself know those characteristics, or sense when they change, such as when passengers get in and out.
Again, it is possible to design controllers that do sense at least some of the disturbances, but it is not inherent to the task of control.
A control system can work without making any predictions about anything.
The room thermostat does not know that the sun is shining, nor the cruise control the gradient. A fortiori, they do not predict that the sun will come out in a few minutes, nor that there is a hill in the distance.
It is possible to design controllers that make predictions, but it is not an inherent requirement of the task of control. The fact that a controller works does not constitute a prediction, by the controller, that it will work. I am belabouring this point, because the error has already been belaboured.
But (it was maintained) doesn't the control system have an implicit model, implicit knowledge, and implicitly make predictions?
No. None of these things are true. The very concepts of implicit model, implicit knowledge, and implicit prediction are problematic. The phrases do have sensible meanings in some other contexts, but not here. An implicit model is one in which functional relationships are expressed not as explicit functions y=f(x), but as relations g(x,y)=k. Implicit knowledge is knowledge that one has but cannot express in words. Implicit prediction is an unarticulated belief about the effect of the actions one is taking.
In the present context, "implicit" is indistinguishable from "not". Just because a system was made a certain way in order to interact with some other system a certain way, it does not make the one a model of the other. As well say that a hammer is a model of a nail. The examples I am using, the thermostat and the cruise control, sense temperature and speed respectively, compare them with their set points, and apply a rule for determining their action. In the rule for a proportional controller:
output = constant × (reference - perception)
there is no model of anything. The gain constant is not a model. The perception, the reference, and the output are not models. The equation relating them is my model of the controller. It is not the controller's model of anything: it is what the controller is.
The only knowledge these systems have is their perceptions and their references, for temperature or speed. They contain no "implicit knowledge".
They do not "implicitly" make predictions. The designer can predict that they will work. The controllers themselves predict nothing. They do what they do whether it works or not. Sometimes, in fact, these systems do not work. The thermostat will fail to control if the outside temperature is above the set point. The cruise control will fail to control on a sufficiently steep downhill gradient. They will not notice that they are not working. They will not behave any differently as a result. They will just carry on doing o=c×(r-p), or whatever their output rule is.
I don't know if anyone tried my robot simulation applet that I linked to, but I've noticed that people I show it to readily anthropomorphise it. (BTW, if its interface appears scrambled, resize the browser window a little and it should sort itself out.) They see the robot apparently going around the side of a hill to get to a food particle and think it planned that, when in fact it knows absolutely nothing about the shape of the terrain ahead. They see it go to one food particle rather than another and think it made a decision, when in fact it does not know how many food particles there are or where. There is almost nothing inside the robot, compared to what people imagine: no planning, no adaptation, no prediction, no sensing of disturbances, and no model of anything but its own geometry. The 6-legged version contains 44 proportional controllers. The 44 gain constants are not a model, they merely work.
(A tangent: people look at other people and think they can see those other people's purposes, thoughts, and feelings. Are their projections any more accurate than they are when they look at that robot? If you think that they are, how do you know?)
Now, I am not explaining control systems merely to explain control systems. The relevance to rationality is that they funnel reality into a narrow path in configuration space by entirely arational means, and thus constitute a proof by example that this is possible. This must raise the question, how much of the neural functioning of a living organism, human or lesser, operates by similar means? And how much of the functioning of an artificial organism must be designed to use these means? It appears inescapable that all of what a brain does consists of control systems. To what extent these may be model-based is an empirical question, and is not implied merely by the fact of control. Likewise, the extent to which these methods are useful in the design of artificial systems embodying the Ultimate Art.
Evolution operates statistically; I would be entirely unsurprised by Bayesian analyses of evolution. But how evolution works is not how the evolved organism works. That must be studied separately.
I may post something more on the relationship between Bayesian reasoning and control systems neither designed by nor performing the same when I've digested the material that Steve_Rayhawk pointed to. For the moment, though, I'll just remark that "Bayes!" is merely a mysterious answer, unless backed up by actual mathematical application to the specific case.
Exercises.
1. A room thermostat is set to turn the heating on at 20 degrees and off at 21. The ambient temperature outside is 10 degrees. You place a candle near the thermostat, whose effect is to raise its temperature 5 degrees relative to the body of the room. What will happen to (a) the temperature of the room and (b) the temperature of the thermostat?
2. A cruise control is set to maintain the speed at 50 mph. It is mechanically connected to the accelerator pedal -- it moves it up and down, operating the throttle just as you would be doing if you were controlling the speed yourself. It is designed to disengage the moment you depress the brake. Suppose that that switch fails: the cruise control continues to operate when you apply the brake. As you gently apply the brake, what will happen to (a) the accelerator pedal, and (b) the speed of the car? What will happen if you attempt to keep the speed down to 40 mph?
3. An employee is paid an hourly rate for however many hours he wishes to work. What will happen to the number of hours per week he works if the rate is increased?
4. A target is imposed on a doctor's practice, of never having a waiting list for appointments more than four weeks long. What effect will this have on (a) how long a patient must wait to see the doctor, and (b) the length of the appointments book?
5. What relates questions 3 and 4 to the subject of this article?
6. Controller: o = c×(r-p). Environment: dp/dt = k×o + d. o, r, and p as above; c and k are constants; d is an arbitrary function of time (the disturbance). How fast and how accurately does this controller reject the disturbance and track the reference?