When I was young, I read popular physics books such as Richard Feynman’s QED: The Strange Theory of Light and Matter. I knew that light was waves, sound was waves, matter was waves. I took pride in my scientific literacy, when I was nine years old.
When I was older, and I began to read the Feynman Lectures on Physics, I ran across a gem called “the wave equation.” I could follow the equation’s derivation, but, looking back, I couldn’t see its truth at a glance. So I thought about the wave equation for three days, on and off, until I saw that it was embarrassingly obvious. And when I finally understood, I realized that the whole time I had accepted the honest assurance of physicists that light was waves, sound was waves, matter was waves, I had not had the vaguest idea of what the word “wave” meant to a physicist.
There is an instinctive tendency to think that if a physicist says “light is made of waves,” and the teacher says “What is light made of?” and the student says “Waves!”, then the student has made a true statement. That’s only fair, right? We accept “waves” as a correct answer from the physicist; wouldn’t it be unfair to reject it from the student? Surely, the answer “Waves!” is either true or false, right?
Which is one more bad habit to unlearn from school. Words do not have intrinsic definitions. If I hear the syllables “bea-ver” and think of a large rodent, that is a fact about my own state of mind, not a fact about the syllables “bea-ver.” The sequence of syllables “made of waves” (or “because of heat conduction”) is not a hypothesis; it is a pattern of vibrations traveling through the air, or ink on paper. It can associate to a hypothesis in someone’s mind, but it is not, of itself, right or wrong. But in school, the teacher hands you a gold star for saying “made of waves,” which must be the correct answer because the teacher heard a physicist emit the same sound-vibrations. Since verbal behavior (spoken or written) is what gets the gold star, students begin to think that verbal behavior has a truth-value. After all, either light is made of waves, or it isn’t, right?
And this leads into an even worse habit. Suppose the teacher asks you why the far side of a metal plate feels warmer than the side next to the radiator. If you say “I don’t know,” you have no chance of getting a gold star—it won’t even count as class participation. But, during the current semester, this teacher has used the phrases “because of heat convection,” “because of heat conduction,” and “because of radiant heat.” One of these is probably what the teacher wants. You say, “Eh, maybe because of heat conduction?”
This is not a hypothesis about the metal plate. This is not even a proper belief. It is an attempt to guess the teacher’s password.
Even visualizing the symbols of the diffusion equation (the math governing heat conduction) doesn’t mean you’ve formed a hypothesis about the metal plate. This is not school; we are not testing your memory to see if you can write down the diffusion equation. This is Bayescraft; we are scoring your anticipations of experience. If you use the diffusion equation, by measuring a few points with a thermometer and then trying to predict what the thermometer will say on the next measurement, then it is definitely connected to experience. Even if the student just visualizes something flowing, and therefore holds a match near the cooler side of the plate to try to measure where the heat goes, then this mental image of flowing-ness connects to experience; it controls anticipation.
If you aren’t using the diffusion equation—putting in numbers and getting out results that control your anticipation of particular experiences—then the connection between map and territory is severed as though by a knife. What remains is not a belief, but a verbal behavior.
In the school system, it’s all about verbal behavior, whether written on paper or spoken aloud. Verbal behavior gets you a gold star or a failing grade. Part of unlearning this bad habit is becoming consciously aware of the difference between an explanation and a password.
Does this seem too harsh? When you’re faced by a confusing metal plate, can’t “heat conduction?” be a first step toward finding the answer? Maybe, but only if you don’t fall into the trap of thinking that you are looking for a password. What if there is no teacher to tell you that you failed? Then you may think that “Light is wakalixes” is a good explanation, that “wakalixes” is the correct password. It happened to me when I was nine years old—not because I was stupid, but because this is what happens by default. This is how human beings think, unless they are trained not to fall into the trap. Humanity stayed stuck in holes like this for thousands of years.
Maybe, if we drill students that words don’t count, only anticipation-controllers, the student will not get stuck on “Heat conduction? No? Maybe heat convection? That’s not it either?” Maybe then, thinking the phrase “heat conduction” will lead onto a genuinely helpful path, like:
- “Heat conduction?”
- But that’s only a phrase—what does it mean?
- The diffusion equation?
- But those are only symbols—how do I apply them?
- What does applying the diffusion equation lead me to anticipate?
- It sure doesn’t lead me to anticipate that the side of a metal plate farther away from a radiator would feel warmer.
- I notice that I am confused. Maybe the near side just feels cooler, because it’s made of more insulative material and transfers less heat to my hand? I’ll try measuring the temperature . . .
- Okay, that wasn’t it. Can I try to verify whether the diffusion equation holds true of this metal plate, at all? Is heat flowing the way it usually does, or is something else going on?
- I could hold a match to the plate and try to measure how heat spreads over time . . .
If we are not strict about “Eh, maybe because of heat conduction?” being a fake explanation, the student will very probably get stuck on some wakalixes-password. This happens by default: it happened to the whole human species for thousands of years.
"Then you may think that "Light is arglebargle" is a good explanation, that "arglebargle" is the correct password. It happened to me when I was nine years old - not because I was stupid, but because this is what happens by default. This is how human beings think, unless they are trained not to fall into the trap. Humanity stayed stuck in holes like this for thousands of years."
Okay, but there's one innocent interpretation even here. People learn language, and when we learn language we copy the verbal behavior of other people. Maybe "arglebargle" is a synonym for light in some language, or maybe it's a supercategory of light (a category that includes light among other things). Maybe the teacher is still in the process of explaining to us what arglebargle means and the first step is to say that light is arglebargle - later on the teacher will tell us what else is arglebargle so that we will gradually build a good concept of it but initially we need to retain the point that light is arglebargle while not yet knowing what arglebargle is, because this is a step in learning what arglebargle is. In that case, we're learning new language when we learn that "light is arglebargle". That's innocent, it's not a mistake.
This suggests that the error may not be learning the teacher's passwords per se, as such, but learning the teacher's passwords when we should ideally be learning something else. The context matters.
But the student is the student and therefore ignorant by assumption, so it may in many cases be too much to expect the student to know when it is time to learn the passwords and when it is time to learn something else. If the student experiences academic success only from learning the passwords, then it may be that the student is not at fault, it's the curriculum that is at fault - the teacher.
So the right recommendation may not be to tell the student to stop learning passwords. Passwords are a legitimate thing to learn, sometimes, and the student, being a student, doesn't know ahead of time which times. The right recommendation may be to adjust the curriculum so that only the right kind of learning yields academic success.
That's probably not easy.
Great example. I need to do something similar when teaching children photosynthesis. It's helpful to start just by teaching a definition (when plants turn light into food). Do they need to learn about stomata and leaf anatomy? No, or at least not this year. That's in next year's textbook. If for now they can remember what is 'photosynthesis,' then their comprehension will be aided later.
I think what Yudkowsky is describing can be a problem sometimes when the point is thinking critically, not memorization of terms and definitions. Learning spanish is basically all about guessing the teacher's password. In literature class, when we want to know what the White Whale is a symbol for, we don't want a 'password' type response.