[SEQ RERUN] Science Isn't Strict Enough

MinibearRex

Today's post, Science Isn't Strict Enough was originally published on 16 May 2008. A summary (taken from the LW wiki):

Science lets you believe any damn stupid idea that hasn't been refuted by experiment. Bayesianism says there is always an exactly rational degree of belief given your current evidence, and this does not shift a nanometer to the left or to the right depending on your whims. Science is a social freedom - we let people test whatever hypotheses they like, because we don't trust the village elders to decide in advance - but you shouldn't confuse that with an individual standard of rationality.

Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was When Science Can't Help, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.

Today's post, Science Isn't Strict Enough was originally published on 16 May 2008. A summary (taken from the LW wiki):

Science lets you believe any damn stupid idea that hasn't been refuted by experiment. Bayesianism says there is always an exactly rational degree of belief given your current evidence, and this does not shift a nanometer to the left or to the right depending on your whims. Science is a social freedom - we let people test whatever hypotheses they like, because we don't trust the village elders to decide in advance - but you shouldn't confuse that with an individual standard of rationality.

This feels a bit like beating up a straw man. Every actual scientific professional develops a sense of what problems are worth working on and what approaches are more or less promising. This sort of intuition isn't scientifically provable -- there's no way to know in advance what you'll find -- but people can and do give reasons why they think X is more promising than Y. People value things like elegance, simplicity, ease of use, and so forth. Learning these sorts of judgements is one of the major things people do as PhD students and junior researchers. It may not be part of the scientific method, strictly defined, but it's something we deliberately teach apprentice scientists.

You can formalize those technical judgements in terms of Solomonof priors and expected utilities if you like, but doing so is a little silly. Different people have different computational hardware and therefore different measures of complexity. Saying "X has a lower Kolmogorov complexity than Y, for me", is no more or less objective than "X seems simpler".

There's also something a little silly about saying "Science isn't good enough, use Bayes". General Bayesian updating is intractable. So you can't use it. All you can ever really do is crude approximations. I don't think you gain a lot by dressing up your judgement in a mathematical formalism that doesn't really do any work for you.