Matt_Simpson comments on Error detection bias in research - Less Wrong

54 Post author: neq1 22 September 2010 03:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (36)

You are viewing a single comment's thread. Show more comments above.

Comment author: CronoDAS 22 September 2010 03:27:21AM 29 points [-]

Feynman once talked about this specific issue during a larger speech:

We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It's a little bit off, because he had the incorrect value for the viscosity of air. It's interesting to look at the history of measurements of the charge of the electron, after Millikan. If you plot them as a function of time, you find that one is a little bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher.

Why didn't they discover that the new number was higher right away? It's a thing that scientists are ashamed of--this history--because it's apparent that people did things like this: When they got a number that was too high above Millikan's, they thought something must be wrong--and they would look for and find a reason why something might be wrong. When they got a number closer to Millikan's value they didn't look so hard. And so they eliminated the numbers that were too far off, and did other things like that. We've learned those tricks nowadays, and now we don't have that kind of a disease.

Comment author: Matt_Simpson 22 September 2010 03:22:00PM *  11 points [-]

We've learned those tricks nowadays, and now we don't have that kind of a disease.

I always thought Feynman was overly optimistic here. Maybe it's true for physicists though.