= 27d567bc2d48d9f40e9ccc20ea370b15)
RichardKennaway comments on Without models - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (53)
A collective reply to comments so far.
All the posted answers to the exercises so far are correct.
1. Warming the thermostat with a candle will depress the room temperature while leaving the thermostat temperature constant.
2. Pressing the brake when the cruise control does not disengage will leave the car speed constant while the accelerator pedal goes down -- until something breaks.
3. The effect of raising a piece-rate worker's hourly rate will depend on what the worker wants (and not on what the employer intended to happen).
4. The doctor's target will be met while patients will still have to wait just as long, they just won't be able to book more than four weeks ahead. (This is an actual example from the British National Health Service.)
Does no-one want to tackle 5 or 6? Anyone who knows the derivative of exp(a t) knows enough to do number 6.
Thank you, kpreid, for linking to the very article that I knew, even while writing the original post, I would be invoking in response to the comments. Anyone who has not come across it before, please read it, and then I will talk about the concept that (it turns out) we are all talking about, when we talk about models, except for the curious fit that comes over some of us when contemplating the simple thermostat.
i77: As you say, the Smith predictor contains a model, and the subsystem C does not. Likewise the MRAC. In the PID case, the engineer has a model. But don't slide from that to attributing a model to the PID system. There isn't one there.
Vladimir_Nesov, pretty much all the concepts listed in the first three sections of that article are special cases of what is here meant by the word. As for the rest, I think we can all agree that we are not talking about a professional clothes horse or a village in Gmina Pacyna. I don't believe I have committed any of these offences (another article I'd recommend to anyone who has only just now had the good fortune to encounter it), but let those call foul who see any.
So, what are we talking about, when we talk about models? What I am talking about -- I'll come to the "we" part -- I said in a comment of mine on my first post:
and more briefly in the current post:
This is exactly what is meant by the word in model-based control theory. I linked to one paper where models in precisely this sense appear, and I am sure Google Books or Amazon will show the first chapters of any number of books on the subject, all using the word in exactly the same way. There is a definite thing here, and that is the thing I am talking of when I talk of a model.
This is not merely a term of art from within some branch of engineering, in which no-one outside it need be interested. Overcoming Bias has an excellent feature, a Google search box specialised to OB. When I search for "model", I get 523 hits. The first five (as I write -- I daresay the ranking may change from time to time) all use it in the above sense, some with less mathematical content but still with the essential feature of one thing being similar in structure to another, especially for the purpose of predicting how that other thing will behave. Here they are:
"So rather than your model for cognitive bias being an alternative model to self-deception..." (The model here is an extended analogy of the brain to a political bureaucracy.)
"Data-based model checking is a powerful tool for overcoming bias" (The writer is talking about statistical models, i.e. "a set of mathematical equations which describe the behavior of an object of study in terms of random variables and their associated probability distributions.")
"the model predicts much lower turnout than actually occurs" (The model is "the Rational Choice Model of Voting Participation, which is that people will vote if p times B > C".)
"I don't think student reports are a very good model for this kind of cognitive bias." (I.e. a system that behaves enough like another system to provide insight about that other.)
The 5th is a duplicate of the 2nd.
Those are enough examples to quote, but I inspected the rest of the first ten and sampled a few other hits at random (nos. 314, 159, 265, and 358, in fact), and except for a mention of a "role model", which could be arguable but not in any useful way, found no other senses in use.
When I googlesearch LW, excluding my own articles and the comments on them, the first two hits are to this, and this. These are also using the word in the same sense. The models are not as mathematical as they would have to be for engineering use, but they are otherwise of the same form: stuff here (the model) which is similar in structure to stuff there (the thing modelled), such that the model can be used to predict properties of the modelled.
In other words, what I am talking about, when I talk about models, is exactly what we on OB and LW are all talking about, when we talk about models, every time we talk about models. There is a definite thing here that has an easily understood shape in thingspace, we all call it a model, and to a sufficiently good approximation we call nothing else a model.
Until, strangely, we contemplate some very simple devices that reliably produce certain results, yet contain nothing drawn from that region of thingspace. Suddenly, instead of saying, "well well, no models here, fancy that", the definition of "model" is immediately changed to mean nothing more than mere entanglement, most explicitly by SilasBarta:
Or the model in the designer's head is pointed to, and some sort of contagion invoked to attribute it to the thing he designed. No, this is butter spread over too much bread. That is not what is called a model anywhere on OB or LW except in these comment threads; it is not what is called a model, period.
You can consider the curvature of a bimetallic strip a model of the temperature if you like. It's a trivial model with one variable and no other structure, but there it is. However, a thermometer and a thermostat both have that model of the temperature, but only the thermostat controls it. You can also consider the thermostat's reference input to be a model of the position of the control dial, and the signal to the relay a model of the state of the relay, and the relay state a model of the heater state, but none of these trivial models explain the thermostat's functioning. What does explain the thermostat's functioning is the relation "turn on if below T1, turn off if above T2". That relation is not a model of anything. It is what the thermostat does; it does not map to anything else.
Exercise 7. How can you discover someone's goals? Assume you either cannot ask them, or would not trust their answers.
This forum has a wonderful feature that allows us to respond to individual comments, generating threads within the discussion that focus on a particular aspect of the topic. Using this feature would be a much better alternative to a single long comment separated from the various comments it refers to.
As I was making a single point in response to many comments, I made the judgement that to say it once in a single place was preferable to splitting it up into many fragments.
No, you made several separate points: 4 responses to 4 solutions to 4 of your exercises, a specific response to i77, your view that some commenters are trying to define away the issues you point out (which could have been a response to Vladimir_Nesov's comment), and your straw man summary of the idea that the controllers are a reflection of the rational process that produced them and your unsubstantiated rejection and don't even indicate which comments you are responding to. This is not just for the benefit of the commenters you respond to, but for those who are following, and may join, the discussion.
There are quite a few different objections to your assertion that control systems work arationally, and your attempt to make a blanket refutation for all of them is unconvincing. In particular, I think you are glossing over the argument that control systems are produced by rational processes by lumping it in with the attempts to redefine models.
Perhaps my prolixity has obscured the substance. Here is a shorter version. The claims are:
1. The concept of a model is entirely unproblematic in this forum.
2. In that entirely unproblematic sense, neither a thermostat nor a cruise control contains a model.
3. The designer of a control system has a model. That model is located in the designer. He may or may not put a model into the system he designs. In the case of the thermostat and the cruise control, he does not.
I shall not repeat all of the evidence and argument, only summarise it:
1. Evidence was given in exhausting detail that I, and we on OB/LW, and the books all mean exactly the same thing by a model. Only in the threads on my two postings on control systems have some people tried to make it mean something different. But changing the definition is irrelevant to the truth-value of the original assertions. I think that no-one is disputing this now, although I shall not be surprised to see further expansions of the concept of a model. (I look forward to SilasBarta's promised article on the subject.)
2. Except for trivial models (one scalar "modelling" another) that leave out what the controller actually does (i.e. control something), there is nothing in either of these controllers but a simple rule generating its output from its inputs. That rule is not a model of something else. It acts upon the world, it does not model the world.
3. That the designer has a model is agreed by everyone. For some reason, though, when I say that the designer has a model, as I have done several times now, people protest that the designer has a model. We are in violent agreement. As for it being in his head, where else does he keep his thinking stuff? Well, "in his head" was not accurate, he might also make a computer simulation, or a physical mock-up. But when it comes time to build the actual system, what he builds is the actual system. The designer models the system; the system does not model the designer.
As for the appropriate form of my response, my judgement on that differs from yours. I shall stop at noting this meta-level disagreement.
What I have observed is that you say that it is not important that the designer has a model, because that model is not part of the control system, and we protest that it is important that the designer has a model, because without that designer and his model, the control system would not exist.
You claimed in your previous article that control systems succeed arationally, though you do not list that claim here. Do you now agree that by following rules produced rationally by an outside agent, the control system is using rationality (indirectly) to succeed?
No. The control system exists because of someone's rational process. Once it exists, it does not work by means of that process. When completed and installed, its operation is screened off from that earlier process. It works only by means of what the designer put into it, not how the designer did that.
The distinction of levels is important. Faced with a control system, to understand how it works it is not necessary to know the designer's thinking, although it may be illuminating in a looking-up-the-answer-in-the-back-of-the-book sort of way. It is only necessary to examine the controller. It is easy to confuse the two, because both the designer and the controller are goal-seeking entities, and there is some overlap between their goals: what the controller controls, the designer designed it to control. But what each does to that end is different.
The distinction is especially important in the case of systems created by evolution, not by a Designer. It is the same distinction that was made between maximising fitness (what the evolutionary process does) and performing the resulting adaptations (what the individual organism does).
You might as well say then that a rationalist only succeeds by his actions, not by the process of choosing those actions, since performing those same actions for any reasons would result in the same success. However, the reasons for those actions are important. Systematic success requires systematically choosing good actions.
A rationalist will often encounter a familiar situation, and without rebuilding his model or recalculating expected utility for various actions, will simply repeat the action taken previously, executing a cached result. This is still rational. Despite the fact that the rationality occurred much earlier than the action it caused, it still caused that action and the resulting success. Notice, using rationality does not necessarily mean going through the rational process. In this case, it means using the results that were previously produced by rationality.
A thermostat follows rational rules, despite being incapable of generating rational rules or even evaluating the effectiveness of the rules it follows. If it were completely "screened off" from the rationality that produced those rules, it would lose access to those rules. You might consider it partially "screened off" in that the rational process does not update the thermostat with new rules, but the initial rules remain a persistent link in the causal chain between rationality and the thermostat's success. I will consider a thermostat's success to be arational when it is actually produced arationally.
As for evolved control systems, evolution is a crude approximation of evidence based updating. Granted, it does not update deep models that can be used to predict the results of proposed actions. It simply updates on propositions of the form that a given allele contributes more to reproductive fitness than alternatives, as represented by the allele frequencies in the population. The crudeness of the approximation and the lack of more advanced rationality features explain why the process is so slow, but the weak rationality of the approximation explains why it works eventually. And the success of evolved control systems owes the effective rules they follow to this weak rationality in the process of evolution.
I don't think the two of you disagree about any actual thing happening when a person designs a thermostat and sets it to run, or when a homeostatic biological system evolves. You only disagree about how to use a certain word.
Well, then let me taboo the issue of whether we call the control systems arational and present my position that I have been arguing for.
Control systems are systematically correlated to features of their environment, particularly the variable they control and their mechanisms for manipulating it. This correlation is achieved by some sort of evidence processor, for example, evolution or a deliberative human designer. This explains why out of the space of possible control systems, the ones we actually observe tend to be effective, as well as why control systems can be effective without processing additional evidence to increase their correlation with their environment.
Perhaps RichardKennaway could follow the same taboo explain his position that the success of control systems indicates a problem with the importance we place on Bayescraft.
From a computer programmer's perspective, a model is something that reflects the state of something else -- even a trivial single value like "the current temperature" or "the desired temperature".
If a thermostat only had a desired-temperature knob or only a "current temperature" indicator, I might agree that there's no model. A thermometer and a control knob don't "model" anything, in that there is nothing "reflecting" them. In the programming sense, there's no "view" or "controller".
But the moment you make something depend on these values (which in turn depend on the state of the world), it's pretty clear in programming terms that the values are models.
Re. exercise 3, Homo economicus will in general work more hours, as working has now become more valuable relative to other uses of her time. In reality, as you say, anything could happen, depending on the individual's utility function for money.
Re. number 6, I've taken too many DEs classes to be excited by this problem, but in general a critically damped system will recover optimally from perturbations.
Re. number 7, one good way is to perturb their behavior in various ways by making them offers (ideally orthogonal ones) and observing their reactions.
"Exercise 7. How can you discover someone's goals? Assume you either cannot ask them, or would not trust their answers."
I'd guess that the best way is to observe what they actually do and figure out what goal they might be working towards from that.
That has the unfortunate consequence of automatically assuming that they're effective at reaching their goal, though. So you can't really use a goal that you've figured out in this way to estimate how good an agent is at getting to its goals.
And it has the unfortunate side effect of ascribing 'goals' to systems that are way too simple for that to be meaningful. You might as well say that the universe has a "goal" of maximizing its entropy. I'm not sure that it's meaningful to ascribe a "goal" to a thermostat - while it's a convenient way of describing what it does ("it wants to keep the temperature constant, that's all you need to know about it"), in a community of people who talk about AI I think it would require a bit more mental machinery before it could be said to have "goals".
It is not about contagion. The point is, the reason that a particular control system even exists, as a opposed to a less effective control system or no control system at all, is that a process that implements some level of rationality produced it. The fact that a control system only needs the cached results of past rationality, and does not even have the capacity to execute additional rationality, does not change the fact that rationality plays a role in its effectiveness.
Semantics check: I assert that evidence accumulation does not imply some (non-zero) level of rationality. Ex gratia, evolution by natural selection accumulates evidence without any rationality. Does my word use accord with yours?
I think of the process of rationality as using evidence to (on average) improve behavior in the sense of using behaviors that produce better results. Evolution is a strange example, in that this process of improvement is not deliberative. It has no model, even metaphorically, that is deeper than "this gene contributes to genetic fitness". It is incapable of processing any evidence other than the actual level of reproductive success of a genetic organism, and even then it only manages to update gene frequencies in the right direction, not nearly the rationally optimal amount (or even as close as deliberative human rationality gets). It is this small improvement in response to evidence that I consider rational (at a very low level). The fact that we can trace the causal steps of the evidence (reproductive fitness) to the improvement at a deep physical level matters only as much as the fact that in principle we could do the same with the causal steps of evidence I observe influencing the neurons in my brain which implements my rationality.
...so that's a "no," then? (I don't think we disagree about what is actually (thought to be) happening, only on the words we'd use to describe it.)
That is correct. We are using the word differently. What do you mean by "rationality"?
That's a question with a complicated answer, but for the purposes of distinguishing what natural selection does from Cyan::rationality, it involves actions that are planned with an eye to constraining the future.
FYI: I'm working on a reply to this, which is becoming long enough and broad enough that I'm going to submit it for the front page.