asr comments on How to teach to magical thinkers? - Less Wrong

14 Post author: polymathwannabe 24 February 2014 01:43PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (79)

You are viewing a single comment's thread. Show more comments above.

Comment author: polymathwannabe 24 February 2014 04:50:07PM 1 point [-]

The techniques of the scientific method are universally valid; they're not contingent on a specific culture. If civilization was wiped today and we had to start from scratch, we would discover the same methods to ascertain natural laws and apply them to our purposes.

When we find an alien culture, I expect they will follow the same rules to find out what works and what is real (if they're advanced enough to use science).

Comment author: asr 25 February 2014 03:30:10AM *  5 points [-]

The techniques of the scientific method are universally valid; they're not contingent on a specific culture. If civilization was wiped today and we had to start from scratch, we would discover the same methods to ascertain natural laws and apply them to our purposes.

Different scientific communities have different methods. The scientific method as practiced by physicists isn't the same method we use in computer systems research and it isn't the same method they use in medical research. And this isn't because these different fields have different deviations from the One True Method -- it's because different subjects require different methods to prevent error.

  • In computer systems and physics, data is typically collected by machines. We therefore aren't worried about observer bias or placebo effects, and we don't usually worry about blinding things from experimenters.
  • With computer systems, everything is reasonably deterministic and so statistical error isn't a major concern. Also, any effect that's big enough to be interesting is likely to be far larger than statistical noise -- it's not an interesting paper unless you got a factor of two improvement or something like that.
  • In physics, it's routine to circulate preprints before a peer-reviewed paper. This doesn't happen much in computer science.
  • In physics and CS, a purely theoretical argument without data can be taken seriously, and often people will trust theory more than experiment. My impression is that people don't have nearly the same sort of confidence in theory in biological or social science.

I think talking about The scientific method is mostly an oversimplification. I don't hear professional scientists using that category when talking amongst themselves about their work. I hear much more about the particular publication and review norms of individual fields.

Comment author: RichardKennaway 25 February 2014 12:03:53PM 4 points [-]

By scientific method I would mean something on a far more general level than details about circulation of preprints.

Architecture varies, but the structural mechanics that describes how buildings stay up is the same always and everywhere.

Comment author: asr 26 February 2014 03:57:51PM 1 point [-]

The awkwardness is that once you generalize enough to cover everything we normally refer to as "science", it's hard to include a very wide range of things we don't normally think of as science.

  • We don't think of legal reasoning as science, but it involves using information and experimentation (with a community of experts!) to update our model of the world.
  • The fashion industry uses experiment and empirical reasoning to figure out what people want to buy. But I don't think it's useful to talk about fashion designers as scientists.

I think the term "scientific method" as normally used in English does not pick out any actual cluster of behaviors or practices. It's a term without a coherent referent.

Comment author: RichardKennaway 26 February 2014 04:13:31PM *  -1 points [-]

I think the term "scientific method" as normally used in English does not pick out any actual cluster of behaviors or practices.

The term "scientific method" as ordinarily used is associated with the traditional rituals of "Science", which are themselves unsatisfactory, or at best an improvable-upon approximation to what really works in finding out about the world. The more useful cluster is the one hereabouts called Bayesian epistemology. It can and should be practiced everywhere, and if a fashion designer employs it, it is just as useful to call it that as when a scientist in the laboratory does.