Fake Explanations

Eliezer Yudkowsky

LESSWRONG
LW

Mysterious Answers

173 Fake Explanations

by Eliezer Yudkowsky

20th Aug 2007

2 min read

173

You are viewing a version of this post published on the 20th Aug 2007. This link will always display the most recent version of the post.

Once upon a time, there was an instructor who taught physics students. One day she called them into her class, and showed them a wide, square plate of metal, next to a hot radiator. The students each put their hand on the plate, and found the side next to the radiator cool, and the distant side warm. And the instructor said, Why do you think this happens? Some students guessed convection of air currents, and others guessed strange metals in the plate. They devised many creative explanations, none stooping so low as to say "I don't know" or "This seems impossible."

And the answer was that before the students entered the room, the instructor turned the plate around.

Consider the student who frantically stammers, "Eh, maybe because of the heat conduction and so?" I ask: is this answer a proper belief? The words are easily enough professed—said in a loud, emphatic voice. But do the words actually control anticipation?

Ponder that innocent little phrase, "because of", which comes before "heat conduction". Ponder some of the other things we could put after it. We could say, for example, "Because of phlogiston", or "Because of magic."

"Magic!" you cry. "That's not a scientific explanation!" Indeed, the phrases "because of heat conduction" and "because of magic" are readily recognized as belonging to different literary genres. "Heat conduction" is something that Spock might say on Star Trek, whereas "magic" would be said by Giles in Buffy the Vampire Slayer.

However, as Bayesians, we take no notice of literary genres. For us, the substance of a model is the control it exerts on anticipation. If you say "heat conduction", what experience does that lead you to anticipate? Under normal circumstances, it leads you to anticipate that, if you put your hand on the side of the plate near the radiator, that side will feel warmer than the opposite side. If "because of heat conduction" can also explain the radiator-adjacent side feeling cooler, then it can explain pretty much anything.

And as we all know by this point (I do hope), if you are equally good at explaining any outcome, you have zero knowledge. "Because of heat conduction", used in such fashion, is a disguised hypothesis of maximum entropy. It is anticipation-isomorphic to saying "magic". It feels like an explanation, but it's not.

Supposed that instead of guessing, we measured the heat of the metal plate at various points and various times. Seeing a metal plate next to the radiator, we would ordinarily expect the point temperatures to satisfy an equilibrium of the diffusion equation with respect to the boundary conditions imposed by the environment. You might not know the exact temperature of the first point measured, but after measuring the first points—I'm not physicist enough to know how many would be required—you could take an excellent guess at the rest.

A true master of the art of using numbers to constrain the anticipation of material phenomena—a "physicist"—would take some measurements and say, "This plate was in equilibrium with the environment two and a half minutes ago, turned around, and is now approaching equilibrium again."

The deeper error of the students is not simply that they failed to constrain anticipation. Their deeper error is that they thought they were doing physics. They said the phrase "because of", followed by the sort of words Spock might say on Star Trek, and thought they thereby entered the magisterium of science.

Not so. They simply moved their magic from one literary genre to another.

Anticipated ExperiencesSemantic StopsignRationality

Frontpage

173

Expecting Short Inferential Distances

106 comments368 karma

Guessing the Teacher's Password

98 comments282 karma

Mentioned in

131Occam's Razor

127Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think

117Disputing Definitions

107Scissors Statements for President?

100The Beauty of Settled Science

New Comment

96 comments, sorted by

oldest

Click to highlight new comments since: Today at 9:11 AM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]Anonymous_Coward17y462

Well, one difference between "heat conduction" and "phlogiston" is that the former carries some additional information with it - heat conduction is a well-understood mechanism by which energy is transferred from place to place. Maybe it does apply in that situation and maybe it doesn't - in the example given, it doesn't, there's no heat-conduction mechanism to transfer heat from one side to the other - but the fact that there's actually a mechanism behind the words separates it, qualitatively, from an explanation like "phlogiston." It has equations behind it which can then be written down and tested for agreement with reality.

Really, I can quite understand the students... if you say "I don't know" you have a zero percent chance of getting the explanation right. If you say "that seems impossible," then you're guaranteed to get it 100% wrong - since it DID happen, and thus it must be possible. The best course of action in the situation is to think of all the hypotheses you can, and then guess at one of them - whichever one has the highest chance of being right, given what they know about physics.

Now, I certainly hope that the student... (read more)

-5MrPineapple14y

[-]orthonormal14y350

Everyone agrees that the physics students are just doing what they've been incentivized to do in class after class. It's just worth pointing out that the behavior they've been trained to do is not at all like doing science, and that nobody seems to know or worry about this.

[-]Eliezer Yudkowsky17y161

AC, what you're describing here is a severe case of déformation educationnelle.

Really, I can quite understand the students... if you say "I don't know" you have a zero percent chance of getting the explanation right.

If you say "I don't know" you have a zero percent chance of getting a gold star in the idiot damned school system. But it is still the rational thing to say when, in fact, you don't know. You can easily do worse than maximum entropy if you guess at random.

Furthermore, "getting it right" by guessing the verbal phrase the teacher has in mind, even if the school system gives you a gold star for it, does not necessarily mean that you possess any anticipation-controllers. All you got right was a string of words, like guessing the passphrase to the teacher's login.

"Heat conduction" is a verbal phrase which may, for someone who knows the equations, invoke genuinely explanatory equations from memory. And for someone who knows the equations, it should be obvious that the equations do not predict the further side being warmer.

If you don't know the equations, then "heat conduction" is a verbal phrase invoking magic from the Sta... (read more)

[-]royf12y100

You are overstating the case by a large margin.

[Saying "I don't know"] is still the rational thing to say when, in fact, you don't know.

Saying "I don't know" may be, to a large degree, the true state of your belief when you use probability theory. But in this case it's not the rational thing to say when you use decision theory. "I don't know" is true, but it is a non-answer to the question, and doesn't get you points. It's a different matter whether this point system is effective or moral, but as long as it's there, that's what you play by.

You can easily do worse than maximum entropy if you guess at random.

If you base your guess correctly on an incomplete model of reality, which you've constructed correctly from past observations, you can never do worse, on average, than maximum entropy. More evidence can never lead to less information (as per the Data Processing Inequality).

Furthermore, "getting it right" [...] does not necessarily mean that you possess any anticipation-controllers.

On the contrary, it mean exactly that. Being rewarded for predictive powers improves your model of the world, whereas "I don't know" is an e... (read more)

4Chrysophylax12y

I disagree. The proper response to not knowing the answer is to admit to not knowing and then give your best guess, not to try to hide your ignorance, because if you succeed then the teacher doesn't know you need help. A student who is more concerned with not displaying ignorance than with not being ignorant is not trying to learn, which is not rational. That which can be destroyed by the truth should be, and it probably won't be if you try to avoid finding out what the truth is. The key phrase here is "on average". If you guess at random from all possible explanations of a given phenomenon, you will, on average, die before guessing the correct answer. There is a reason the monkeys with typewriters are given infinite time to reproduce Hamlet. Moreover, as the set of answers considered increases in size, the expected utility from giving any one answer tends towards the expected utility of a wrong answer. As long as giving the wrong answer gives less utility than admitting ignorance, admitting ignorance is almost always the utility maximising option if you don't know. If I write down a number and then take a number from a table of random numbers, and the numbers are the same, does this mean that I'm psychic? Because if getting the correct answer means that I have useful anticipation controllers then I must be. "I don't know" is not an excuse for not knowing. That makes no sense whatsoever. "I don't know" is a statement about whether I know something or not, not a statement about whether I ought to know. If you can't admit fallibility then you will never learn anything. The points you make about the benefits of testing students' knowledge are true. Unfortunately, they miss the point - while it is important not to penalise guessing incorrectly, so as not to dissuade admitting ignorance, it is much better to actively reward admitting that you have tried and failed. If a confused student does not always seek an explanation, the reward for seeking explanations isn't l

0V_V12y

If students could always get away with an "I don't know" they wouldn't have much incentive to learn anything. More importantly, the school system main purpose is not to teach you just a collection of facts. It has to teach you how to behave in the world, where you often have to make choices based on incomplete information.

5wedrifid12y

0 marks for "I don't know". 1 mark for a correct answer. -1 mark for an incorrect answer. Not only is it a simple incentive system I've done exams that implemented similar systems. (Westpac math competition for example.)

2V_V12y

That is a sensible scoring system which is in fact widely used.

-8Chrysophylax12y

0Decius12y

Allow both an answer and a certainty. -x points for an incorrect answer with certainty x +2x points for the correct answer with certainty x Alternately, +10^x points for a correct answer with certainty x, and +Log(1-x) points for the incorrect answer. This encourages an attempt to answer every question, even if the certainty is rated as 0.

4Jiro10y

Yes, I know, old post. If you give the student -X points for an incorrect answer with certainty X, and +2X points for a correct answer with certainty X, the expected value of giving an answer and lying about its certainty as Y is (1-X)(-Y) + (X)(2Y) = 3XY - Y. If X is less than 1/3, the student should lie and claim that his certainty is 0, while if X is greater than 1/3, he should lie and claim that his certainty is 1. I'm not going to try to find the maximum for the second version, but it should be obvious that the student is still better off lying about his true certainty. Of course, you could just avoid telling the student how you're going to grade, but the score will then just depend on how well the student guesses your grading criteria.

0Decius10y

Neither of my described systems are ideal. Squared error works for binary questions, but it would reward "Pi is exactly 3, with 0 confidence". Rather than allow continuous estimates of accuracy, I think that the ideal system would ask the student to provide a range of confidence, (five choices from "guessing" to "Certain", with equivalent probabilities), and an appropriate scoring rule; a guess would be penalized 0 for being wrong but gain little for being right, and going from "almost certain" to "certain" would add a small value to a correct answer but a large penalty to a wrong answer. Having established the +points for correct and -points for wrong for each confidence description, do the math to determine what the actual ranges of confidence are, sanity check them against the descriptions, and then tell the student the confidence intervals. (Alternately, pick the intervals and terms and do the math to figure out the + for correct answer and -for incorrect answer for those intervals.)

0Jiro10y

It's hard to come up with a system where the student doesn't benefit from lying about his certainty. What you describe would fix the case from 4 (almost certain) to 5 (certain), but you need to get all the cases to work and it's plausible that fixing the 4 to 5 case (and, in general, increasing the incentive to pick 4) breaks the 3 to 4 case. After all, you can't have all the transitions between certainty values add a small value to a correct answer. You must have a transition where a large value is added for a correct answer and your system may break down around such transitions.

0Decius10y

The largest value would be added for the first confidence interval, which would also add the smallest cost to being wrong with that confidence.

0Jiro10y

That would mean a large value would be added when going from "guess" to "almost guess", which would mean that it would be beneficial for a student to lie and claim to almost guess when he's really completely guessing.

1Decius10y

Suppose the student thinks that there is a 10% chance that he is right, and the reward structure is +5/-1 for confidence interval 1. In fact, make the reward structure:(right/wrong) 1/0, 6/-1, 10/-3, 13/-6, 15/-10, 16/-15 That puts the breakpoints at roughly even intervals, keeps the math easy, and with a little bit of clarifying exactly where the breakpoints are, doesn't reward someone who accurately determines their accuracy and then lies about it.

6Jiro10y

I sat down late last night trying to prove that this couldn't work and instead proved that it could. If I did this correctly, in order for it to work, with the confidences increasing from 0 to 1, left side confidence <= (difference in Y)/(difference in X + difference in Y) right side confidence >= (difference in Y)/(difference in X + difference in Y). Differences in X are 5, 4, 3, 2, 1 and differences in Y are 1, 2, 3, 4, 5 leading to values of 1/6 through 5/6; as 0 < 1/6 < 1/5 < 2/6 < 2/5 < 3/6 < 3/5 < 4/6 < 4/5 < 5/6 < 1 this is immune to lying within a single interval (and also turns out to be so for multiple intervals).

2Decius10y

So, what are the downsides of making this a grading standard? The biggest one I see is that it would be unfair except in classes that have as prerequisites an outstanding score in a class that covers credence calibration.

5Chrysophylax12y

Students who do not care about education do get away with not knowing anything. Detention is not much of a punishment when you don't show up. It is difficult to prevent a student who cares deeply about eduction from admitting ignorance, since admitting ignorance is necessary in asking for explanations. The difficult task is persuading students who care about doing well to seek knowledge, rather than good marks. These students are not motivated enough to learn of their own accord - they never volunteer answers or ask questions openly, because they care more about not being thought ignorant (or, of course, keen) than about not being ignorant. The point is not to allow students to "get away with" admitting ignorance. There is a vast difference between not knowing the answer and not wanting to know. Personally, I have never found it hard to tell the difference between students who don't want to know and students who don't want to be judged by their peers. It is very rarely a bad idea to publicly admit that you might be wrong, especially when you are guessing. A school that does not teach the importance of separating your beliefs and your ego has failed miserably. Whatever else it has taught, it has not taught its students how to learn.

1whowhowho12y

How true

[+]Alexis_Perrier17y-160

[-]ed_johnson17y80

I agree with AC...you're being too hard on the students. I doubt very much they were stating anything with confidence. It's quite possible that some of them didn't really care about understanding physics and were just trying to get the right answer to please the teacher, but others were probably just thinking out loud. Thinking "maybe it's heat conduction" might just be the first step to thinking "no, it can't be heat conduction," or even to realizing "I don't really understand heat conduction," and there is nothing wrong w... (read more)

[-]DilGreen14y160

I think that EY's problem with this point of view is a typical one that I find here at LW: a consideration of the rational thinker as loner in heroic mode, who is expected to ignore all contexts (social, environmental, whatever) that are not explicitly stated as part of the problem presentation. On the other hand, these students were in a physics class, and the question is obviously not part of normal conversation.

-2matteyas7y

Are you saying that in an environment for learning about- and discussing rationality, we should strive for a less-than-ideal rationality (that is, some form of irrationality) just because of practical contexts that people often run into and choose the easy way out of? Would you become equally suspicious of the math teacher's point of view if some person from a math problem buys 125 boxes with 6 watermelons each, since he won't be able to handle that amount in most practical contexts?

[-]Senthil17y10

Ed, the student's response may be due to something he needs to unlearn as discussed in the following earlier post:

http://lesswrong.com/lw/i2/two_more_things_to_unlearn_from_school/

If it's not the case, that is, if he doesn't need to unlearn anything he may still be incorrect in his understanding. In that case, this post tells one of the reasons why he may be incorrect and be aware of it.

I think this is worse compared to the behaviour addressed in the earlier post.

[-]Stuart_Armstrong17y150

but that was entirely rational because the professor set them up to believe that.

They were rational, but not unbiased. They wanted to maximise their chances of pleasing the prof., not maximise their chances of understanding the world.

I think this teaching approach was great, and I might use something similar myself (there are mathematical equivalents of the above situation). Learning science means that you have to learn a boatload of facts, and learn the scientific method. Since the boatload of facts has to be accepted without question (for the whole of your early career), this undermines the teaching of the method (when it is taught at all). A few sessions like this (properly exploited by the instructor) would do a world of good.

8johnlawrenceaspden12y

Hmm, the boatload of facts (and the theories behind them) explain a larger boatload of facts that you already know. I found physics and maths very clear and easy and exciting because of this. Despite my first love in infancy being chemistry, I abandoned it when they wanted me to memorise the colours of the transition metal ions. If they'd told me instead how those colours came about from the quantum theory, and shown me all the pretty colours by actually burning the damned things, instead of turning them into despicable rote work, I might have grown up to be a chemist.

[-]Hopefully_Anonymous17y10

Great post, Eliezer, and I agree with Stuart. There should be no valor in stating an uncertain guess as a certain statement -one should at least express one's level of uncertainty.

[-]Hopefully_Anonymous17y20

Incidentally, this is an area where legal instruction is superior to scientific instruction at the graduate/pre-thesis level.

6gwern14y

How so?

0AtreidesOne8y

Perhaps due to the presumption of innocence? They are constantly aware that you have to have proof beyond reasonable doubt to convict someone, whereas in other fields we are more likely to assume an answer exists?

5johnlawrenceaspden12y

A bit late, but I'd like an answer to this too. I enjoyed talking to law students at college. They clearly have the same sort of minds as mathematicians, except they can also talk like proper humans do, and their problems are interesting. If they have a better way of teaching that sort of thing, maybe someone should try using it for science.

6Phoenix_Wright12y

I think perhaps the reason one would say that legal education is better is that it is understood from the first day that many of the problems that will be posed actually have no answer ("What is justice?" "How can we balance the interests in this scenario?" "What would the reasonable [sic] person do given this dilemma?") and that what is important is the quality of the reasoning you use to come to your answer, not the outcome. When a well-argued, incorrect answer is scored more highly than a correct answer with no justification, the get-the-gold-star incentive is removed and it improves quality of thought on the matter. Maybe this person meant something entirely different; I can't claim to speak on their behalf.

3Desrtopa12y

It would be no surprise if legal training turns out clever arguers, but there's a big difference between arguing persuasively and getting the right answer. Training in questions where there is no single right answer may improve students' rhetoric, but I think it's likely to leave them underprepared when they have to weigh in on questions where there is a single right answer, and no amount of argument will make any other answer acceptable.

[-]conchis17y130

"They wanted to maximise their chances of pleasing the prof., not maximise their chances of understanding the world."

I don't know that I buy this. If the students make a guess that's wrong, one would expect that to kickstart a process of the professor helping them to understand why it's wrong. (Student: "Um... because of heat conduction?" Teacher: "OK, what does heat conduction suggest should happen in this situation?"...) This seems more likely to result in learning than just sitting there and saying "I don't know". If anything, I think it's often a bigger problem from a learning perspective, when people are too afraid of being wrong to put out tentative ideas.

"I don't know" is a rational response to this situation if you are sure enough of your understanding of all the potential principles involved that you know they can't explain the phenomenon (and you don't happen to guess that the professor is messing with you). But it's fairly clear the students aren't in that situation, so starting to generate hypotheses about what's going on seems perfectly sensible. Of course, they should be actual hypotheses, and Eliezer's perfectly right... (read more)

4DSimon11y

How about "I don't know, but maybe it has something to do with X?"

[-]Byrne17y60

Is there any way to set up a classroom (or an educational system) so that these students would get the right answer? Alternatively, is it even desirable?

If you teach students to think this way, you're saying "The world is governed by comprehensible scientific laws -- which is irrelevant, because people are constantly screwing with you." This experiment might be useful in a physics class for lawyers (who would probably catch on) or conspiracy theorists (who would, at least, have more entertaining hypotheses).

A compromise might be for the teacher t... (read more)

6Tyrrell_McAllister13y

You are teaching them that, if they understand the scientific laws, they can catch the people constantly screwing with them.

5MixedNuts13y

Let's test this. Now. Who has access to a group of creationists, a group of evolutionists, a room with a heater and a metal plate?

[-]Eliezer Yudkowsky17y100

I should note that I read about this scenario in the Canonical List of Science Jokes but I have no idea whether it was a note from someone's experience. If anyone tries this, I'd love to know the result - my guess is that in real life, at least one student in the class would guess it, which is why I'd suggest having students write down their best suggestions on paper; followed by the teacher asking "How could you falsify your theory?" and writing that down as well.

Explaining things by magic has been the default state of human existence for far longer than science ever existed. Anyone using fancy words must be assumed to be invoking magic by default.

The training of a rationalist must be strict! No human can be unfair enough; you have to match swords against Nature to develop the requisite skills.

[-]Bob317y10

I would like to re-emphasize Eliezer's point that "I don't know" (not an uneducated guess) is the proper answer to any question where, in fact, a student (or person in general) does not know the right answer, with the addition of "but I will find out." On my exams a (fully) incorrect answer gets zero points while "I don't know but I will find out" gets one-half credit. You still fail if you don't know anything, but at least you are not in an idiot damned school system. Rewarding students for data dumps when they don't know an answer cannot be healthy. Or maybe I'm just biased because my approach makes exam grading significantly easier....

4gjayb13y

Out of curiosity: do you mean that you give students credit for professing that they will find out? Or do you have them take the problems home sometime later, look things up, do the work, and then give them half the points back? Because I have seen the latter work very well, while I would see the former as once again asking students to put down what their teachers want to hear.

4johnlawrenceaspden12y

That's a wonderful idea. Timed exams but you can get (tiny amounts) of extra credit for handing in the answers later in the day having asked your friends. Screws it up as an intelligence test (which may be the real point of formal exams), but would do wonders for learning if you did it on regular informal tests.

1Viktor Riabtsev6y

My favorite thing to do in physics/math classes, all the way up 2nd year in university (I went into engineering), was to ask others how they fared on tests, (in order to) then try to figure out why my answers were wrong. I found genuine pleasure in understanding where I went wrong. Yet this seemed taboo in highschool, and (slightly less) frowned upon in university. I feel like rewarding the student who messed up, however much or little, with some fraction of the total test score, like 10%; would be a great idea. You gain incentive to figure out what you missed; even if you care little about it. That's better then nothing.

[-]conchis17y20

Bob, Unless guessing is part of finding out. (This clearly isn't the case in an exam, but often is in a classroom situation.)

[-]Robin_Hanson217y10

Eliezer, I hope you are considering writing a book based on this excellent series of essays you have been writing.

[-]Eliezer Yudkowsky17y00

Conchis, the problem is guessing passphrases instead of anticipation-controllers.

Robin, I am indeed considering it, it will depend on how much raw material I can generate.

[-]Shakespeare's_Fool17y90

A student who said it was done by magic would, of course, have been correct. Because it was done by magic.

The teacher moved the plate when the audience wasn't looking. That is one of the ways magicians perform their tricks.

If they had used words such as "supernatural," "miracle," or "paranormal," then they would not have been discussing physics.

But good magicians are the best practical physicists.

[-]Senthil17y-10

Eliezer, are you also considering giving free copies of the book to people who frequent this blog? :-)

[-]conchis17y00

Eliezer, Agreed. (That was my original point.)

[-]beowulflovesgrendel17y20

The larger experiment seems to me to be the teacher's looking for someone with an answer to the ENTIRE 'experiment' which includes a 'false' set up. This isn't about 'physics,' it's about overall discernment, much the same way a truly observant participant will 'see through a magic trick,' no mean feat. So, for me, "I don't know" is the only honest and complete answer. It denotes an empty glass which (at least) can be filled and restricts that answwer to a particular observer and does not make 'unknowable" a universal state. Answers which a... (read more)

[-]beowulflovesgrendel17y00

regretting the above typos

[-]ron17y10

I find it difficult to believe that none of the students would have guessed that the plate was turned around.

5Benevolence12y

Or is this just hindsight bias? Edit: im a fool, new to posting on LW, just noticed the date. Point still stands though (not that i expect a reply)

6DaFranker12y

You are not a fool... or so I want to believe anyway. The Welcome to LessWrong page tells us that it's fine to just resume discussion even on old posts if you think there's something to add, and that sometimes new discussions started this way can be worth more than "not wasting your time" replying to old comments. Though of course, if you want a reply from the original author of the comment, you might want to first check if that user is still around, I reckon.

1Benevolence12y

Ahh, thank you for the link DaFranker.

[-]Wyrd_Fuquad16y120

At one of the websites I frequent the first paragraph of this article was posted.

I guessed the teacher had set up the plate and turned it around.

As a student of the theatre I am somewhat versed in the arts of "Illusion". There was one show where we set off a smoke bomb and lowered the lights at the end of the first act. We then had intermission At the beginning of the second act we set of a smoke bomb and raised the lights. It's amazing how many people wrote about how we made the prop appear out of nowhere on the stage. They edited the intermission out of their remembrance of the story.

This story quickly sprang to my mind and I realized that if you can fool someone into forgetting an intermission you can fool someone that had no idea what was done before they came into the room in the first place.

[-]Todd_Non-Coward16y30

Eliezer, I have to disagree with some of what you wrote. The question was phrased as to give what you thought would be an answer. In such a setting "I don't know" would not be something to say -- silence would be the result. Additionally, if someone said "heat conductance?" they're not saying they DO know, but it's clearly a guess... and possibly without the question mark in their tone, they really are throwing it out as a guess with little or no confidence, while not believing with any confidence that it's the reason. Additionally, the... (read more)

[-]Dmytry15y20

Phew. This is just an 'ask a stupid question, get a stupid answer' situation.

Questions 'why' and 'what is' are metaphysical or semantic and have nothing whatsoever to do with science. The only reason why those are prevalent in education is that education sucks. Science is not a search for answers to "why X happens", even though it is popularized as such.

My school physics wasn't like this at all (eastern Europe here). I would have a problem to solve - how many watts of heating are required to maintain uniform temperature of 20 degrees Celsius in a... (read more)

3royf12y

I agree that the word "why" is perhaps not the best choice, but it really stands for "what is the physical mechanism for". Anyway, what you describe is engineering, not science. A scientist explains the mechanism by which a stone ax can kill a mammoth. An engineer uses this understanding to design a better ax. A craftsman builds the ax. A warrior takes the ax, kills the mammoth, and wins the girl.

2johnlawrenceaspden12y

And yet without them, science barely seems worth doing. I mean, someone finds out a new and seemingly useless fact about gravity and their name goes down in history. Someone makes a slight improvement to a mobile phone and they're not even famous in their own company.

[-]DilGreen14y220

Interestingly enough, my teacher, Chris. Alexander (author of A Pattern Language), recounts his entrance test for a physics degree at Cambridge. The applicants were asked to experimentally determine the magnetic field of the earth. He performed the experiment, and came up with an answer he knew to be wrong. Wrong by too large a margin to put down to experimental error. A smart chap, he had time to repeat the key part of the experiment, and recalculate - got the same answer. He used the last part of his time to write down his hypothesis for having achieved such a result. And, alone among the students, he was right. A massive electro-magnet was being used on the floor below as part of another experiment.

I believe the advice offered to me as an 18yr old physics student encountering similar circumstances was simply to show my workings and the incorrect result, and to add that I knew this was not the 'right' answer.

4Vladimir_Nesov13y

I had a similar episode in (Russian equivalent of) 10th grade, where a physics class lab experiment had critically flawed equipment, but we were supposed to write down all the steps according to a predetermined script described in the textbook. I instead described what was really happening in the experiment, why, and what was different from the intended scenario. The teacher marked other students according to how well they adhered to the script, even though it didn't square with the actual experimental data in any way, and they had to forge or "reinterpret" the data. (I did get an A, but possibly only because of my prodigy status.)

[-]Vladimir_Nesov13y70

I have a bad memory for isolated facts (like names or past events), and comparatively poor ability to guess intended meaning of things people are talking about in real time. When I was younger, I would just randomly guess possible answers to patch over the gaps in my mental picture (with little chance for actual success) in situations where it was expected of me to know the answer. Generation of random explanations that have nothing to do with actually explaining the observations might sometimes be motivated by a similar psychological pressure to give some... (read more)

[-]Douglas_Reay13y30

I not so sure that when the student suggests "because of heat conduction", they are attempting to provide a full explanation.

I model their internal thinking more along the lines of "Well, I don't know for sure what's going on here, this is an obscure effect I've not come across before, but it seems plausible to me that it will be in some way connected with conduction, so I'll suggest that as a first step, and hope someone else can fill in the mathematical details for me."

It is closer to the situation when a company owner says to her man... (read more)

0johnlawrenceaspden12y

hmm. surely that would affect sales first?

[-]johnlawrenceaspden12y40

I heard this in the popular 'Oxbridge Interview Question' genre, a long time ago. It actually makes great sense there, as a 1960s don would have had a coal fire burning in winter, when the interviews are done, and be able to turn the plate round between interviews. And the interviewer would be expecting everyone to know all the relevant laws, and be looking for exactly the right level of bewildered confusion and hypothesis generation that you're hoping for.

The point is not to guess the right answer (that's essentially random inspiration and the ability to ... (read more)

[-]aspera12y10

Unless I misunderstand, this story is a parable. EY is communicating with a handwaving example that the effectiveness of a code doesn't depend on the alphabet used. In the code used to describe the plate phenomenon, “magic” and “heat conduction” are interchangeable symbols which formally carry zero information, since the coder doesn't use them to discriminate among cases.

I’m sincerely confused as to why comments center on the motivations of the students and the professor. Isn't that irrelevant? Or did EY mean for the discussion to go this way? Does it matter?

0chaosmosis12y

You'll quickly find that LessWrongers love tangents and digressions.

5wedrifid12y

EY can mean whatever he wants. He gets to choose what is in the post, everyone else gets to choose what they would like to talk about.

2JonMcGuire11y

People focus on the motivations of the students and the professor because the professor's behavior is unorthodox. The students paid good money to learn about physics. As others have mentioned, you can't be too hard on them, they arrive at class expecting a physics lesson, not sleight-of-hand. Consequently, my initial response to the article was that I understood what EY meant to convey, but I thought there were probably other ways to illustrate it that didn't involve the unnecessary "trickery" demonstrated by the professor. However, upon further reflection, the professor's trickery itself could be characterized as relevant to EY's point. If we completely ignore the proferred "magic explanations" from the students, one might consider the professor's trick a lesson that all the physics education in the world may be inadequate to explain a puzzling observation. In other words, I found it helpful to assume that the professor was also trying to make a point similar to that which EY was making, instead of assuming that the professor just felt like being a jerk that day. As a bonus, by focusing on the conditions of the scenario instead of just the answers, a student who is smart enough to recognize that their education may be inadequate could still answer "I don't have enough information to explain this," which implies he still believes there is an explanation, which might be a better answer than just "I don't know," which sounds a lot like just giving up.

[-]jafrumstra12y40

"Because of heat conduction" is the correct answer-heat from the radiator conducts to the plate-heat from the plate conducts to the air regardless of which side is being examined. The teacher asked "Why does this happen?" not "Why is the closest side to the radiator cold and the distant side hot?"-the question which is assumed to be implied by the actual one. The answer to THAT question would be "because of magic" since the professor was performing an illusion that was prepared ahead of time. The data points necessar... (read more)

[-]RogerS12y00

Many different things can be deduced from this story, as previous comments have illustrated. The step that I question is "carries no information" = "magic". I prefer Karl Popper's account, in which [to paraphrase "Conjectures & Refutations" Chapter 1] "carries no predictive information" = "metaphysical" but "metaphysical" does not mean "unscientific". Rather, science involves two activities, hypothesis creation and hypothesis testing. It is the hypothesis testing that has to be exclu... (read more)

[-]A1987dM11y00

If you say "heat conduction", what experience does that lead you to anticipate? Under normal circumstances, it leads you to anticipate that, if you put your hand on the side of the plate near the radiator, that side will feel warmer than the opposite side.

Metals conduct heat much faster than air, so if conduction was the only mechanism responsible you'd expect both sides of the plate to be at nearly the same temperature.

[-]Mati_Roy11y00

Would you consider "black energy" as a fake explanation for the expansion of the universe?

[-]Jiro10y20

(Response to old post)

These are students, so they don't have perfect understanding of science. Even if they understand how to calculate what some theories predict, they don't know exactly when to apply those theories or what confounding effects might occur.

So unlike someone with perfect understanding, they don't know with 100% certainty that any specific theory applies. Asking what caused X to happen is really asking "what theory, among the ones you know, has the highest probability of having caused this result".

But even if the result is wrong... (read more)

[-]Slvador10y10

I have seen this example before. I actually do not blame the students at all for the following reasons (some taken from other comments)

1) They are thinking out loud, so seeing that some aspects points it could be heat conduction(after all that would be the typical reason for most temperature discrepancies withing an item) then they scream "heat conduction" as an invitation for closer look which is a valid (as pointed by other commentors) method of thinking

2) They are screaming the highest probable answer they can think of. Magic and heat conducti... (read more)

3Basil Marte6y

I've not seen anybody mention those students who said "strange metals in the plate" in particular, and I'd like to argue for them. Their answer was not a password (the teacher never mentioned it), and actually shows correct anticipation-controlling beliefs! That is to say, they noticed that the observed outcome is not what they would have predicted, and looked for some hypothesis that explained why the heat gradient is reversed. Working from the incorrect assumption that they are seeing a stationary state, they guessed a hidden means of transporting heat from the heater to the far end of the plate, e.g. an insulated internal layer. I might be overfitting on this few-word detail, but I think this answer is on average very early in search orderings of those with qualitative but correct visualization of heat conduction, and I don't see other equally simple reasons why the students would have said this. Of course, I also agree with the rationalist point that this answer should still feel a bit forced, trigger a listing of assumptions, which on average hits "stationary state" very early for simple physics problems.

[-]Yosarian28y10

I was just re-reading the sequences, and I have to say that as a teacher I really think you're misjudging what is happening here.

Much of learning, it seems, is building up a mental framework, starting from certain concepts and attaching new concepts to them so that they can easily be recalled later and so you can use the connections between concepts to develop your own thinking later..

From my point of view, it looks like the student (perhaps as long as a year ago) had successfully created a new concept node in their mind, "heat conduction". Th... (read more)

1Wes_W8y

I'm not sure you've described a different mistake than Eliezer has? Certainly, a student with a sufficiently incomplete understanding of heat conduction is going to have lots of lines of thought that terminate in question marks. The thesis of the post, as I read it, is that we want to be able to recognize when our thoughts terminate in question marks, rather than assuming we're doing something valid because our words sound like things the professor might say.

0Yosarian28y

Yeah, that's fair, although it sounds like the student he's quoting did understand that. I'm just saying that "guessing the teacher's password" isn't usually a fair way to view what's going in in cases like this. "Building up a concept map of connections between related concepts" is probably more accurate, and that really is a vital part of the learning process, it's not a bad thing at all.

[-]DreamFlasher8y40

We can be swayed by the context we are operating in, thinking inside-the-box
Don’t use terms and explanations if you are not really sure about the concepts

[-]Pod5y10

I can't find the quoted joke in scijokes.txt, can anyone help?

[-]MoritzG5y10

When you are presented with a very unlikely outcome you have to accept it.

Had the teacher shown a dozen dice all showing the same number and asked how he did it, there would have been two answers:

2. You

[This comment is no longer endorsed by its author]Reply

[-]MoritzG5y10

Had the teacher presented a dozen of dice all showing the same number and asked how this could have happened they would have been wiser.
But the situation is similar. In pure theory this could happen naturally, in that case doubting it would be a case of gamblers fallacy or not knowing the Anthropic principle.

If you encounter the impossible you should check your assumptions, but to say that a human like entity has caused this outcome is dangerous.

[-]MichaelDickens2y20

Perhaps it's worth distinguishing between two types of "I don't know":

I don't know because I haven't put any thought into it. (This is the type of "I don't know" that teachers rightly discourage.)
I don't know because I have considered several hypotheses, and none of them explain my observations. (For example, my mental model of heat conduction predicts that the close side of the plate should be hotter, not the far side, so that explanation fails.)

Perhaps teachers should encourage students to replace "I don't know" with "my mental model predicts A, but I observe B", which communicates that the student is thinking correctly about the problem.

[-]titotal1y120

Note that the nearer side feeling colder than the farther side is be completely possible.

The key is that they didn’t check the temperatures of each side with a thermometer, but with their hands. And your hands don’t feel temperature directly, they feel heat conduction. If you have a cake and a baking tin that are the same temperature, the metal will feel hotter because it is more conductive.

If I wanted to achieve the effect described here without flipping trickery, I might make the side near the radiator out of a very nonconductive plastic (painted to look like metal), and the side further away out of a very thermally conductive metal. It seems entirely plausible to me that once everything is in thermal equilibrium, the plastic would feel cooler than the metal, despite having a higher temperature. Of course, this would depend on the actual temperatures, conductivities, etc involved. But in principle it seems completely possible.

Suddenly the physics students don’t look so foolish. In fact, in this scenario, both the heat conduction guy and the “strange metals in the plate” guy are on the right track! (even the convective guy could have a point, if that's the primary meth... (read more)

1Firinn1y

....I'm a little mindblown by reading this, honestly, because I read 'Fake Explanations' when I was like eleven years old, and I really felt like it changed the way I thought and was extremely influential on me at that early point in my life, and I kept telling people this story, and also I never thought of this, and now I am strongly negatively updating against my own success at internalising the lessons here. I guess the lesson from this is that the correct answer isn't "it's really obvious that the instructor flipped the plate around and the students should have realised this as soon as they Noticed They Were Confused", but "when you encounter confusing information, you should feel comfortable remaining confused until you have actually spent some time generating more hypotheses and learning more information". The answer of "the plate was flipped around" is semi-obvious (it's often the first hypothesis generated by smart people when I recount this story to them) and we all... stopped thinking at that point, and patted ourselves on the back for being so rational? This feels a bit like it deserves to inspire a top-level post along the lines of "there is a second higher-level version of the Fake Explanations post, which points out that weird metals is a possible explanation, and if you laughed at the guy who was considering the possibility of the plate being made out of some sort of weird material and considered yourself superior for not being so stupid, then you should feel bad and go reread the stuff about motivated stopping". Or something.

2Said Achmiz1y

I agree that “it’s really obvious that the instructor flipped the plate around and the students should have realised this as soon as they Noticed They Were Confused” is not the correct answer. However, I think that your suggested lesson isn’t quite right either—namely, the “until…” clause is superfluous. Now, I can’t speak for physics students, at whatever level of physics education the students in the story were at… but for myself, I don’t think I could’ve generated the hypothesis outlined in the grandparent. (Or, perhaps more precisely, maybe I could’ve generated at least approximately that hypothesis—but only alongside a number of other hypotheses which would be physically implausible/inapplicable/etc.) In other words: there is no way I could’ve solved the puzzle (without first learning much more physics, which presumably is outside the scope of the problem). And this, in my experience, happens often. There is some phenomenon, and you don’t know the explanation for it; there is some mystery, and you don’t know the solution to it. And the rational conclusion is that you aren’t going to figure out the answer. You just don’t know. You can spend some large amount of time or effort learning and developing expertise in the relevant domain, certainly! But you’re not going to figure out the right answer by thinking about it, because the space of possibilities for what the answer could be includes any number of unknown unknowns: things that you aren’t aware of, and that you don’t know you’re not aware of. Thus the rational response is to follow the wisdom of Homer [edit: actually it was Bart] Simpson: can’t win, don’t try. You don’t know, and you can’t figure it out, and that’s all there is to it. Either invest the considerable effort needed to research the subject matter in general and the problem in particular, or simply stop at “I don’t know”.

1Firinn1y

I'm not a physics student, but I absolutely feel I should have been able to generate more than one hypothesis here! I have definitely enjoyed watching science videos that talk about really cool ceramics that get used in building spacecraft, which can be glowing red-hot and nevertheless safe to touch because of how non-conductive they are. So it's not like I wasn't aware of the possibility that some materials have weird properties here. It's just that I generated a single hypothesis - the instructor flipped the plate around - and was super-satisfied with being correct. And maybe I get Bayes points for being correct, since "the instructor flipped the plate around" is the right answer (assuming it's a real story) and "the instructor went to all the trouble of constructing a two-sided plate out of really weird materials purely in order to fuck with his physics students" is a wrong answer. But where I think I went wrong is feeling derisive towards the silly incorrect physics students who say things like "maybe something weird is happening with heat conduction?" and feeling superior to them because they were just guessing the teacher's password. When, actually, "something weird is going on relating to conduction" is a thought which could have led to me generating and considering more than one hypothesis.

2Noosphere891y

This seems like a You Are Not Measuring What You Think Are Measuring moment. Link below: https://www.lesswrong.com/posts/9kNxhKWvixtKW5anS/you-are-not-measuring-what-you-think-you-are-measuring

[-]Ernesto G.1y10

There are a few key aspects to consider here:
1 The instructor is an authoritative figure. Students assume the question posed by the instructor is one with merit and that he isn't playing tricks on them.
2 This is a physics class, not a philosophy class? If it was the second one, students may be inclined to think outside the box. But nobody expects a physics teacher to be playing tricks on the experiments.
3 Students will always try to give their best guess if they're pressed for an answer. "I don't know" is rarely an acceptable answer by any teacher.

Moderation Log

96Comments