Today's post, Einstein's Arrogance was originally published on 25 September 2007. A summary (taken from the LW wiki):
Albert Einstein, when asked what he would do if an experiment disproved his theory of general relativity, responded with "I would feel sorry for [the experimenter]. The theory is correct." While this may sound like arrogance, Einstein doesn't look nearly as bad from a Bayesian perspective. In order to even consider the hypothesis of general relativity in the first place, he would have needed a large amount of Bayesian evidence.
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was How Much Evidence Does It Take?, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
I don't think this addresses what Eliezer's point actually it is. The point is that the general region of evidence that causes one to locate a hypothesis and not have a hideously large amount of evidence for it is a pretty small region.
I think Eliezer is incorrect here, but the problem with the argument is subtle and requires careful examination of different historical discoveries.
Let's look at three historical examples and one example unfolding now.
First, let's look at Kepler's discovery that the orbits of the planets were ellipses. Given the very accurate data Kepler had access to thanks to Tycho Brahe's careful examination Kepler could see as soon as he looked at the data that ellipses fit really well. Moreover, if one accepted the idea and looked at the orbits one saw this really amazing pattern Kepler's third law which related the orbits of different planets to each other, an incredible display of an apparent relationship between the different planets, something the like of which had never been seen before in any other model. We all love to talk about Galileo but the truth is that it was Kepler's work which was the death blow for other theories. It not only was mathematically simple (ok, it used the most advanced math of the time but didn't have the hundreds of adhoc parameters that other models like the various Ptolemaic models, the Copernican model, Tycho's model, or the many hybrid models floating around at the time (some people wanted Venus or Mercury to orbit the sun. This is rarely discussed now as part of the history even though it was one of the most common views. Note that this is due in part because Galileo decided not to actually address this approach at all. So talking about it doesn't fit in with the amazing Galileo narrative). So once one saw the data one immediately knew it worked.
Next, consider Newton's discovery of gravity. Again, this required the most advanced math of the time. But, Newton was able to show that if one believed his laws of gravity one got Kepler's results and a lot of other known results (such as an approximate (but incomplete) description of the tides) so that once one had the hypothesis in the right form it became obvious. Oresme as early as the 1300s had discussed a law of gravity that diminished as objects got farther away, but he didn't really do much with it (he never specified the rate of decline with distance) so it really was hitting on the right function that did it. Moreover, Newton's laws explained observed very minor discrepancies in Kepler's laws (planets sped up or slowed down their orbits sometime when they were very near each other), and further observation and calculation quickly confirmed it. Still, this was only suggestive due in part to the very severe difficulty with measuring such minute differences. So it wasn't until the discovery of a whole new planet based on Newton's laws that Newtonian gravity got to be really accepted.
Now's let go to another field: chemistry. The idea that combustion took something from air was around in some form about the same time that phlogiston was proposed. It is common here to describe phlogiston as an example of an unfalsifiable theory. But this is unfair. The more extreme, unfalsifiable versions of phlogiston arose at the way end of the period when Joseph Priestly, actually credited by many as the discoverer of oxygen, refused to give up the phlogiston theory to his dying day and continued to add ad hoc hypotheses to defend it even as the rest of chemistry had gone on. (This is one of the very few examples where the whole old-scientists-need-to-just-croak thing actually happened. That pseudo-Kuhnian meme is very common but it isn't a very good model for reality.) (Excellent book on Priestly is Steve Johnson's "The Invention of Air") But, when phlogiston was actually proposed, it and the idea that fire took something from the air really did have close to the same amount of evidence. It was only over time as the careful work of measuring mass in reactions (which took a lot of careful, technical work when the reactions involved gases, or even worst, sometime multiple different gases) gradually produced a new theory which included oxygen and other elements. Yet unlike the Newton and Kepler examples, this not only happened gradually. the observed patterns didn't fit into some nice theory. The number of elements seemed to be massive, and their exact behavior seemed to have little rhyme or reason. It wouldn't be until the late 19th century that anyone had more than an inkling of what was happening. So the understanding of the basics of combustion arose as evidence gradually painted the picture in and gradually became more and more clear that the oxygen theory was essentially correct.
Now, let's look at one more example: Dark matter. This is an interesting one because there are many different hypotheses to explain what we don't see. A lot of the work has been simply eliminating the more easily falsifiable hypotheses, or doing extremely careful work eliminating difficult to falsify hypotheses (for example, it has taken a lot of extremely sensitive experiments to determine that the three known neutrino flavors can't contribute to more than 1-5% or so of the observed dark matter discrepancy). We have a lot of different ideas. Some of which are mathematically very elegant (here I'm counting different versions of supersymmetry or other proposed extensions of the Standard Model as different theories, and also including some of the stranger ideas like the hypothesis that part of the problem is that we are using Newtonian or nearly Newtonian approximations of GR that don't quite due what we want in some contexts). Will someone come up with a sudden idea that makes everything work nicely and explains dark matter and lots of other stuff besides just as SR explained multiple apparently unconnected problems at once? Maybe, but I doubt it. I suspect that what is more likely is that eventually one of the theories or even a collection of theories (say for example some version of supersymmetry and a lot more machos hanging around and a few sterile neutrinos) will gradually emerge as dealing with the discrepancy. This doesn't fit Eliezer's model.
So what is the upshot? Sometimes things go the way Eliezer depicts, and for much of the history of physics and astronomy this was a really good approximation. But it doesn't always go that way and it especially doesn't go that way if one is looking at fields other than physics.
Ah. I was misinterpreting what Eliezer meant. Thank you.