Followup to: What is control theory?

I mentioned in my post testing the water on this subject that control systems are not intuitive until one has learnt to understand them. The point I am going to talk about is one of those non-intuitive features of the subject. It is (a) basic to the very idea of a control system, and (b) something that almost everyone gets wrong when they first encounter control systems.

I'm going to address just this one point, not in order to ignore the rest, but because the discussion arising from my last post has shown that this is presently the most important thing.

There is a great temptation to think that to control a variable -- that is, to keep it at a desired value in spite of disturbing influences -- the controller must contain a model of the process to be controlled and use it to calculate what actions will have the desired effect. In addition, it must measure the disturbances or better still, predict them in advance and what effect they will have, and take those into account in deciding its actions.

In terms more familiar here, the temptation to think that to bring about desired effects in the world, one must have a model of the relevant parts of the world and predict what actions will produce the desired results.

However, this is absolutely wrong. This is not a minor mistake or a small misunderstanding; it is the pons asinorum of the subject.

Note the word "must". It is not disputed that one can use models and predictions, only that one must, that the task inherently requires it.

This is long, but it's the shortest length I could cut from the material and have a complete thought.

1. Alien Space Bats have abducted you.

In the spirit of this posting, I shall describe a magical power that some devices have. They have an intention, and certain means available to achieve that intention. They succeed in doing so, despite knowing almost nothing about the world outside. If you push on them, they push back. Their magic is not invincible: if you push hard enough, you may overwhelm them. But within their limits, they will push back against anything that would deflect them from their goal. And yet, they are not even aware that anything is opposing them. Nor do they act passively, like a nail holding something down, but instead they draw upon energy sources to actively apply whatever force is required. They do not know you are there, but they will struggle against you with all of their strength, precisely countering whatever you do. It seems that they have a sliver of that Ultimate Power of shaping reality, despite their almost complete ignorance of that reality. Just a sliver, not a whole beam, for their goals are generally simple and limited ones. But they pursue them relentlessly, and they absolutely will not stop until they are dead.

You look inside one of these devices to see how it works, and imagine yourself doing the same task...

Alien Space Bats have abducted you. You find yourself in a sealed cell, featureless but for two devices on the wall. One seems to be some sort of meter with an unbreakable cover, the needle of which wanders over a scale marked off in units, but without any indication of what, if anything, it is measuring. There is a red blob at one point on the scale. The other device is a knob next to the meter, that you can turn. If you twiddle the knob at random, it seems to have some effect on the needle, but there is no fixed relationship. As you play with it, you realise that you very much want the needle to point to the red dot. Nothing else matters to you. Probably the ASBs' doing. But you do not know what moves the needle, and you do not know what turning the knob actually does. You know nothing of what lies outside the cell. There is only the needle, the red dot, and the knob. To make matters worse, the red dot also jumps along the scale from time to time, in no particular pattern, and nothing you do seems to have any effect on it. You don't know why, only that wherever it moves, you must keep the needle aligned with it.

Solve this problem.

That is what it is like, to be one of these magical devices. They are actually commonplace: you can find them everywhere.

Related to: The Psychological Unity of Humankind, Instrumental vs. Epistemic: A Bardic Perspective

"Everyone generalizes from one example. At least, I do."

   -- Vlad Taltos (Issola, Steven Brust)

My old professor, David Berman, liked to talk about what he called the "typical mind fallacy", which he illustrated through the following example:

There was a debate, in the late 1800s, about whether "imagination" was simply a turn of phrase or a real phenomenon. That is, can people actually create images in their minds which they see vividly, or do they simply say "I saw it in my mind" as a metaphor for considering what it looked like?

Upon hearing this, my response was "How the stars was this actually a real debate? Of course we have mental imagery. Anyone who doesn't think we have mental imagery is either such a fanatical Behaviorist that she doubts the evidence of her own senses, or simply insane." Unfortunately, the professor was able to parade a long list of famous people who denied mental imagery, including some leading scientists of the era. And this was all before Behaviorism even existed.

The debate was resolved by Francis Galton, a fascinating man who among other achievements invented eugenics, the "wisdom of crowds", and standard deviation. Galton gave people some very detailed surveys, and found that some people did have mental imagery and others didn't. The ones who did had simply assumed everyone did, and the ones who didn't had simply assumed everyone didn't, to the point of coming up with absurd justifications for why they were lying or misunderstanding the question. There was a wide spectrum of imaging ability, from about five percent of people with perfect eidetic imagery¹ to three percent of people completely unable to form mental images².

Dr. Berman dubbed this the Typical Mind Fallacy: the human tendency to believe that one's own mental structure can be generalized to apply to everyone else's.

(Disclaimer: This post is sympathetic to a certain subset of theists.  I am not myself a theist, nor have I ever been one.  I do not intend to justify all varieties of theism, nor do I intend to justify much in the way of common theistic behavior.)

I'm not adopted.  You all believe me, right?  How do you think I came by this information, that you're confident in my statement?  The obvious and correct answer is that my parents told me so¹.  Why do I believe them?  Well, they would be in a position to know the answer, and they have been generally honest and sincere in their statements to me.  A false belief on the subject could be hazardous to me, if I report inaccurate family history to physicians, and I believe that my parents have my safety in mind.  I know of the existence of adopted people; the possibility isn't completely absent from my mind - but I believe quite confidently that I am not among those people, because my parents say otherwise.

To begin, here are some Fun Psychology Facts:  

People who were asked to describe a face after seeing it are worse at recognizing the same face later.

People who are asked to describe a wine after drinking it are worse at recognizing the same wine later.

People who are asked to give reasons for their preferences among a collection of jellies are worse at identifying their own preferences among those jellies.

This effect, known as Verbal Overshadowing, occurs primarily when a principally non-verbal process is disrupted by a task which involves verbalization.  The above generalizations (and Verbal Overshadowing effects more generally), do not occur among what we can term "Verbal Experts": individuals who are as good at verbalizing the relevant process as they are at doing it implicitly or automatically.  This seems like it will be very important to keep in mind when cultivating our own Rationality.

(This article expands upon my response to a question posed by pjeby here)

I've seen a few back-and-forths lately debating the instrumental use of epistemic irrationality -- to put the matter in very broad strokes, you'll have one commenter claiming that a particular trick for enhancing your effectiveness, your productivity, your attractiveness, demands that you embrace some belief unsupported by the evidence, while another claims that such a compromise is unacceptable, since a true art should use all available true information. As Eliezer put it:

I find it hard to believe that the optimally motivated individual, the strongest entrepreneur a human being can become, is still wrapped up in a blanket of comforting overconfidence. I think they've probably thrown that blanket out the window and organized their mind a little differently. I find it hard to believe that the happiest we can possibly live, even in the realms of human possibility, involves a tiny awareness lurking in the corner of your mind that it's all a lie.

And with this I agree -- the idea that a fully developed rational art of anything would involving pumping yourself with false data seems absurd.

Still, let us say that I am entering a club, in which I would like to pick up an attractive woman. Many people will tell me that I must believe myself to be the most attractive, interesting, desirable man in the room. An outside-view examination of my life thus far, and my success with women in particular, tells me that I most certainly am not. What shall I do?

Once upon a time, Seth Roberts took a European vacation and found that he started losing weight while drinking unfamiliar-tasting caloric fruit juices.

Now suppose Roberts had not known, and never did know, anything about metabolic set points or flavor-calorie associations—all this high-falutin' scientific experimental research that had been done on rats and occasionally humans.

He would have posted to his blog, "Gosh, everyone!  You should try these amazing fruit juices that are making me lose weight!"  And that would have been the end of it.  Some people would have tried it, it would have worked temporarily for some of them (until the flavor-calorie association kicked in) and there never would have been a Shangri-La Diet per se.

The existing Shangri-La Diet is visibly incomplete—for some people, like me, it doesn't seem to work, and there is no apparent reason for this or any logic permitting it.  But the reason why as many people have benefited as they have—the reason why there was more than just one more blog post describing a trick that seemed to work for one person and didn't work for anyone else—is that Roberts knew the experimental science that let him interpret what he was seeing, in terms of deep factors that actually did exist.

Related to: Joy in the Merely Real, How An Algorithm Feels From Inside, "Science" As Curiosity-Stopper

Your friend tells you that a certain rock formation on Mars looks a lot like a pyramid, and that maybe it was built by aliens in the distant past. You scoff, and respond that a lot of geological processes can produce regular-looking rocks, and in all the other cases like this closer investigation has revealed the rocks to be completely natural. You think this whole conversation is silly and don't want to waste your time on such nonsense. Your friend scoffs and asks:

"Where's your sense of mystery?"

You respond, as you have been taught to do, that your sense of mystery is exactly where it should be, among all of the real non-flimflam mysteries of science. How exactly does photosynthesis happen, what is the relationship between gravity and quantum theory, what is the source of the perturbations in Neptune's orbit? These are the real mysteries, not some bunkum about aliens. And if we cannot learn to take joy in the merely real, our life will be empty indeed.

But do you really believe it?

I loved the Joy in the Merely Real sequence. But it spoke to me because it's one of the things I have the most trouble with. I am the kind of person who would have much more fun reading about the Martian pyramid than about photosynthesis.

And the one shortcoming of Joy in the Merely Real was that it was entirely normative, and not descriptive. It tells me I should reserve my sense of mystery for real science, but doesn't explain why it's so hard to do so, or why most people never even try.

So what is this sense of mystery thing anyway?

Previously in series:  My Way
Followup to:  The Sin of Underconfidence

Good online communities die primarily by refusing to defend themselves.

Somewhere in the vastness of the Internet, it is happening even now.  It was once a well-kept garden of intelligent discussion, where knowledgeable and interested folk came, attracted by the high quality of speech they saw ongoing.  But into this garden comes a fool, and the level of discussion drops a little—or more than a little, if the fool is very prolific in their posting.  (It is worse if the fool is just articulate enough that the former inhabitants of the garden feel obliged to respond, and correct misapprehensions—for then the fool dominates conversations.)

So the garden is tainted now, and it is less fun to play in; the old inhabitants, already invested there, will stay, but they are that much less likely to attract new blood.  Or if there are new members, their quality also has gone down.

Then another fool joins, and the two fools begin talking to each other, and at that point some of the old members, those with the highest standards and the best opportunities elsewhere, leave...

I am old enough to remember the USENET that is forgotten, though I was very young.  Unlike the first Internet that died so long ago in the Eternal September, in these days there is always some way to delete unwanted content.  We can thank spam for that—so egregious that no one defends it, so prolific that no one can just ignore it, there must be a banhammer somewhere.

But when the fools begin their invasion, some communities think themselves too good to use their banhammer for—gasp!—censorship.

After all—anyone acculturated by academia knows that censorship is a very grave sin... in their walled gardens where it costs thousands and thousands of dollars to enter, and students fear their professors' grading, and heaven forbid the janitors should speak up in the middle of a colloquium.

Spreading the Word prompted me to report back as promised.

I have two sisters aged 17 and 14, and mom and dad aged 40-something. I'm 22, male. We're all white and Latvian. I translated the articles as I read them.

I read Never Leave Your Room to the oldest sister and she expressed great interest in it.

I read Cached Selves to them all. When I got to the part about Greenskyers the older sister asserted "the sky is green" for fun. Later in the conversation I asked her, "Is the sky blue?", and her answer was "No. I mean, yes! Gah!" They all found real life examples of this quickly - it turns out this is how the older sister schmoozes money and stuff out of dad ("Can I have this discount cereal?" followed by "Can I have this expensive yogurt to go with my cereal?").

I started reading The Apologist and the Revolutionary to them but halfway through the article they asked "what's the practical application for us?", and I realized that I couldn't answer that question - it's just a piece of trivia. So I moved on.

I tried reading about near-far thing to them, but couldn't find a single good article that describes it concisely. Thus I stumbled around, and failed to convey the idea properly.

In the end I asked whether they'd like to hear similar stuff in the future, and the reply was an unanimous yes. I asked them why, in their opinion, haven't they found this stuff by themselves and the reason seems to be that they have have no paths that lead to rationality stuff in their lives. Indeed, I found OB through Dresden Codak, which I found through Minus, which I found through some other webcomic forum. Nobody in my family reads webcomics not to mention frequenting their forums.

The takeaway, I think, is this: We must establish non-geeky paths to rationality. Go and tell people how to not be suckers. Start with people who would listen to you. You don't have to advertise LW - just be +5 informative. Rationality stuff must enter the mass media: radio, TV, newspapers. If you are in a position to make that happen, act!

I would also like to see more articles like this one on LW - go, do something, report back.