Eugine_Nier comments on Rationality Quotes October 2012 - Less Wrong

8 Post author: MBlume 02 October 2012 06:50PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (298)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 26 October 2012 02:29:05PM 3 points [-]

The best description of the scientific method I have ever seen is from a conceptual physics textbook by Hobson. Paraphrasing:

The scientific method involves the dynamic interplay between theory and experiment.

That’s it. Perfect. As a scientist, I don’t come to work on Monday and make an observation, then form a hypothesis on Tuesday, devise an experiment to test some prediction on Wednesday, perform the experiment Thursday, and interpret the result on Friday. On any given day I would be hard pressed to tell you where I am in the process. All of the above, really. It’s a mess. It’s a constant back and forth comparing theoretical expectations to the final arbiter of any dispute: nature. Some people specialize in one aspect of the process, and can spend years chewing on some piece of it. But it is seldom done in isolation.

Meanwhile, science fair projects across the nation—under the advisement of teachers who themselves often do not have personal experience in how science really works—approach their subject in an uncharacteristically formulaic way. Nine times out of ten the effort culminates in a proof that the initial hypothesis was right; as if that were the goal and criterion for success. The rare student is surprised by the data, admitting to a failure of the hypothesis, quickly reconsidering initial assumptions and driving into an unexpected yet rewarding direction (dynamic interplay). That’s the real scientist at work. Too bad the judges (in my experience as a judge) often don’t recognize this apparent failure as the true success.

I can’t pass up the opportunity to share with you the “best” high school science fair project I ever saw (when I was myself a student participant in the fair—and no, it was not my project): “Does light travel through the dark?” Setup: light-tight cardboard box painted black on the inside; flashlight shining through a hole in one end; a peephole in the other end to see if the light made it. Any guesses?

-- Tom Murphy

Comment author: Eugine_Nier 26 October 2012 11:54:17PM 1 point [-]

As a scientist, I don’t come to work on Monday and make an observation, then form a hypothesis on Tuesday, devise an experiment to test some prediction on Wednesday, perform the experiment Thursday, and interpret the result on Friday. On any given day I would be hard pressed to tell you where I am in the process. All of the above, really. It’s a mess.

That is a large part of the reason why we have problems like the file drawer effect and data dredging.

Comment author: chaosmosis 28 October 2012 08:10:19AM 1 point [-]

I don't think that thinking categorically and mechanically would be feasibly productive.

It's a reality that we have to think messily in order to solve problems quickly, even if that efficiency also causes biases.

Comment author: Eugine_Nier 28 October 2012 11:25:19PM 1 point [-]

However, we should at least be aware of what the proper way to do it would be.

Comment author: chaosmosis 28 October 2012 11:56:09PM *  0 points [-]

Yeah. But I think there are different levels of propriety, and that is what the quote is getting at. We should mention that the ideal form of science would look very rigid and modular and be without bias. Then, we should talk about how actual science inevitably involves biases and errors, and that these biases to a limited extent are sometimes compensated by increased efficiency. Then, we should talk about how to minimize biases while maximizing the efficiency of our thought processes.

Level One: Ideal

Level Two: Reality

Level Three: Pragmatic Ideal

A class or book on Level Three would be very useful to me and I'm not aware of any. Anyone have suggestions? Less Wrong seems to cover Level One very well and Level Two is obvious to anyone who is a human being but Level Three is what I would really like to work on.