MarsColony_in10years comments on Link: The Cook and the Chef: Musk's Secret Sauce - Wait But Why - Less Wrong

3 Post author: taygetea 11 November 2015 05:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (31)

You are viewing a single comment's thread. Show more comments above.

Comment author: ChristianKl 11 November 2015 11:51:40PM 0 points [-]

Yeah, but if you steel-man it,

The act of steelmanning means to argue against a different position then the one the person is holding. It very worthwhile to criticize people for holding positions for the wrong reasons.

I think he was trying to make something similar to a map-territory distinction. It's often useful to make a distinction between the data and our best interpretation of the data. Some conclusions don't require much extrapolation, but others require a great deal.

To me what you are saying doesn't seem like a description of the map-territory distinction. A map is not an extrapolation of the territory but an abstraction of it.

If things are "open questions" until they are above a confidence interval of, say, 0.99

That sentence doesn't look to me like it's inspired by looking at what scientists do. I'm not aware of a scientific community having the standard of question being closed when they are over a confidence interval of 0.99.

You might argue that scientists should do things that way, but that doesn't have much to do with the question of how scientists act in the real world.

It's often useful to make a distinction between the data and our best interpretation of the data.

Statements about what's useful are different than statements that describe what scientists do in reality.

Using a historical example which happens to be false just complicates things. If I recall, philosophers first hypothesized a round earth around 600 BCE, but didn't prove it experimentally until 300 BCE.

I think you missed a point. At 300 BCE they were not centrally concerned with proving via experiment that the earths is round. They instead cared about things making sense intuitively. The idea that it's important to prove claims via experiment came with Descartes into the scientific mosaic which happened much later.

Nobody at the time between 600 BCE and 300 BCE said: “The part of the Earth that I can see at any given time appears to be flat, which would be the case when looking at a small piece of many differently shaped objects up close, so I don’t have enough information to know what the shape of the Earth is. One reasonable hypothesis is that the Earth is flat, but until we have tools and techniques that can be used to prove or disprove that hypothesis, it is an open question.”

If things are "open questions" until they are above a confidence interval of, say, 0.99, then just about everything we discuss here is an open question, as the quote suggests.

"How does action at a distance work?" wasn't an open question shortly after Descartes. It became again an open question when Newton was shown to be right by the expedition that measured the shape of the earth.

In biology the central dogma of molecular biology was considered a close question for a long time. Biologists where confident about the fact that a lot of the DNA is junk DNA that doesn't do anything.

One of the main reasons why we don't consider the question of whether homeopathy works an open question isn't just that we lack empiric evidence for it working but that we based on our theories of chemistry we don't believe that it could work.

Whether or not chiropratics interventions work was a question that scientists considered not to be open for a long time.

Comment author: MarsColony_in10years 12 November 2015 02:11:34AM 0 points [-]

In passing, he gestured vaguely at a vague conception of science. I guess that doesn't qualify as an argument, so perhaps there is no argument to steelman. But I think that the vague conception of science he was trying to gesture toward does correspond to a real thing that scientists sometimes do.

In the map-territory analogy, this might correspond to a fuzzy or blank region of the map. A scientifically minded person might well say "One reasonable hypothesis is that the ~~Earth is flat~~ the blank region looks like nearby regions, but until we have tools and techniques that can be used to prove or disprove that hypothesis, it is an open question."

But here's the idea I think the author was trying to gesture at. In my experience, most people are way too eager to try and solve problems they don't fully understand. I've often heard scientists and engineers caution against this, but the most notable quote is from the rocket scientist Wernher Von Braun: "One good test is worth a thousand expert opinions". I've seen people like Bill Nye repeat this, and seen plenty of science-themed reminders that test results are often surprising, since the world is often much more complex that we give it credit for.

As for the historical commentary, I completely agree. The scenario isn't historically plausible. The scientific revolution would have had to happen earlier just to produce someone capable of saying the quote, and society would have had to somehow go through a scientific revolution without noticing that the earth was round.

Comment author: ChristianKl 12 November 2015 12:06:55PM 0 points [-]

The world is indeed often surprising but frequently surprising in a way that scientific experiments open new questions that weren't in the mind of the scientists beforehand.

When looking at something like Reiki, Bill Nye and friends argue don't consider it an open question whether or not Reiki works just because we don't have well controlled studies investigating it. They consider it not to work because they don't believe that there's ki.

You might argue that they are wrong to do so, but that's still how they operate.

I've seen people like Bill Nye repeat this, and seen plenty of science-themed reminders that test results are often surprising

The phrase science-themed sounds to me more like it refers to science mythology than serious history of science. To the extend that you want to sensible talk about what scientists do, you have to listen to people who study what scientists do and that's not the speciality of a rocket scientist like Wernher Von Braun.

Comment author: MarsColony_in10years 12 November 2015 06:51:04PM 0 points [-]

Ah, that's the definition about which we were talking past each other. I certainly wouldn’t say that "Reiki might work, and until we test it we just don't know!" Perhaps it "works" somewhat through the placebo effect, but even in the unlikely event of a study showing some random placebo controlled health benefit, it would still be astronomically unlikely that ki was the mechanism. (That's not to say that no one will look at the real mechanism after the fact, and try to pick out some superficial similarity to the idea of "ki".)

But that’s beside the point. For hypotheses that are worth our time to test, we test them precisely because it’s an open question. Until we take the data, it remains an open question. (at least for certain definitions of “open question”) I think that’s the point the author was trying to get at with his infeasible historical example.

Comment author: ChristianKl 13 November 2015 02:43:27PM 1 point [-]

If all the available evidence was well explainable with a flat world then the argument that things could also be explained by a very large ball likely wouldn't convince people to consider it an open question. Occam's razor suggests that you go with the most simple theory and not consider more complex theories simply because they fit the data if there are not additional reasons in their favor.

What happen to be open questions depends a lot on the standards of a given academic community. By the standards of mathematics P=NP is an open question. By the standards of biology a similar question where all available evidence points in one direction would be considered closed.

A year ago biology as taught via textbooks didn't consider the question whether or not the lymphatic system extends into the brain to be an open question. The textbooks were clear about there not being a lymphatic system extension into the brain. Now someone found that it extends into the brain and we have to change the textbooks.

Furthermore when the paradigm of a field changes it frequently happens that open questions of the old field get forgotten even through they are not answered.