Great post. In the running for your best ever. "To put a random number on it, I doubt that anything more than one-in-a-million g-factor is required to be a potential world-class genius, implying at least six thousand potential Einsteins running around today." Let's hope it's not one-in-a-hundred-billion. Because then we may never see another potential Einstein.
For the life of me, I don't understand why no one else in the intelligent, persistent-maximizing space wants to start assembly line cloning/breeding our smartest existential risk minimizers besides me (and possibly one other person, I don't know if they're public about it). If this happened in the next few years, a lot of us could benefit from the potential accelerated breakthroughs starting 25-35 years from now. I think smart, open-minded folks (Eliezer, Robin, Aubrey) need to justify to us why they're not pounding the podium on this one NOW, at least anonymously.
Good post, minor nitpicks though:
IIRC, Einstein wasn't the first to try to develop a curvature theory of gravity. Riemann himself apparently tried. And, IIRC, Einstein was one of Riemann's students. Einstein brought to the table the whole thing about having to deal with spacetime rather than space.
As far as Mach's principle, I believe it's something a little different than what you said. It's more the whole thing of acceleration rather than velocity. It's sorta a notion that the inertia of an object derives from the distribution of all the rest of the mass in the universe. I beleive in pure form it's more a vague notion rather than a formalized principle. But it does inform GR and aspects of GR conform to it.
More generally, I'd like to toss out a bit wild guess/wild eyed thought based on this sort of thinking: even if Barbour's timeless universe is false, I'm going to guess that his type of configuration space is the "actual" configuration space. That is, it's a configuration space of relations between particles or whatever, rather than configurations of absolute positions. Now, instead of demanding that the relevant triangle inequalities hold, as he does... don't demand that. Then configurations where the inequality is violated would correspond to curvature. Maybe. Anyways, that's juat a vague notion on my part. If/when I have more of the relevant theoretical sophistication, I'll see if I can make this work, or of it goes kablewey.
Hopefully: nature vs nurture and all that. We probably would want to figure out how to reproduce the relevant training too. Further, that could apply to other people, not just the clones. And that, actually, seems to be what Eliezer is trying to do here in general, actually.
Now, instead of demanding that the relevant triangle inequalities hold, as he does... don't demand that. Then configurations where the inequality is violated would correspond to curvature.
Nope... dS^2 between me and my future self 2 seconds from now is 2 seconds squared. But both have zero dS^2 with a sphere one light second away. No curvature is present.
If on the other hand you're talking about just the space part, then not even curvature can do what you're talking about.
Psy-Kosh, you're copping out with a fake nitpick, in my opinion. Especially with your last two sentences. To avoid threadjacking, I invite you to continue the conversation on my blog.
If we take a statistical analysis of the scientists who tried using Einstein's method, what percentage would have been right? Aristotle was mentioned earlier. You can make a case that Marx and Freud tried using a similar style of reasoning without much success.
Cracking post, really well written. I feel as though this could have preceded the last few and stood up on its own, in which case I may have been more on board recently. But that's by the by - this all rang very true.
So how does one sit down in an armchair, think about what we know about protein folding, and go about solving the big problem? Einstein didn't start by studying AI, probability theory, Bayesian reasoning etc. What did he have that we all seem to find so elusive? There must be a more satisfactory answer than 'a really high g-factor.'
Joseph - don't just make that assertion, draw something from it. As Eliezer says, we should never forget the failures. But bearing in mind the 'quantum fiasco', and thinking about Einstein, it's those great minds producing those great leaps forward that have been responsible for a good deal of those ratchet turns: in particular the most difficult, unintuitive ones. Imagine if we could teach that!
Cosmides & Tooby's writing bugs me. They have a strong, strong tendency to assume you've read whoever they're complaining about and would love to hear about them. Even when they're right, it's the sort of thing that isn't likely to stand the test of time and can be deeply annoying to people who don't care about who they're arguing with.
Given this perspective on what Science does and does not encourage, can you explain the phenomenon of String Theory to us?
Question: where are these great introductions to relativity you speak of? I've had difficulty with the subject thus far.
What did he have that we all seem to find so elusive?A willingness to reconsider his assumptions, an openness to new explanations, and an abiding belief that hypotheses should always be tested against the data - and discarded if they were found wanting.
None of these traits are common, and they are especially rare here.
One way to evaluate a Bayesian approach to science is to see how it has fared in other domains where it is already being applied. For instance, statistical approaches to machine translation have done surprisingly well compared to rule-based approaches. However, a paper by Franz Josef Och (one of the founders of statistical machine translation) shows that probabilistic approaches do not always perform as well as non-probabilistic (but still statistical) approaches. Basically, maximizing the likelihood of a machine translation system produces results that are significantly worse than directly minimizing the error. The general principle is that you should maximize the function that is closest to the criteria that you most care about. Maximizing the probability of a system won't give you good results if what you really care about is minimizing the error.
By analogy, maximizing the likelihood of scientific hypotheses may lead to different results from minimizing the error. Currently, science tries to minimize the error -- it is always trying to disprove bad hypotheses through experimentation. The best hypotheses are the ones left standing. If science switched to maximizing the likelihood of the best hypotheses, this might lead to unintended consequences. For instance, it might be easier to maximize the probability of your pet hypothesis by refining your priors rather than by seeking experiments that could potentially disprove it.
That's an interesting notion. I don't see how Bayesian reasoning is restricted to trying to maximize the likelihood of the 'best' theory'. One of its crowning achievements is to avoid talking just about the best theory and using the full ensemble at all times. You're perfectly free to ask any question of the ensemble. This includes 'Which response minimizes some error function?'
"But sometimes experiments are costly, and sometimes we prefer to get there first... so you might consider trying to train yourself in reasoning on scanty evidence, preferably in cases where you will later find out if you were right or wrong. Trying to beat low-capitalization prediction markets might make for good training in this? - though that is only speculation."
Zendo, an inductive reasoning game, is the best tool I know of to practice reasoning on scanty evidence in cases where you'll find out if you were right or wrong. My view of the game: one player at a time takes the role of "reality", which is a single rule classifying all allowed things into two categories. The other players, based on a steadily growing body of examples of correctly classified things and the fact that the other player made up the rule, attempt to determine the rule first. This is fundamentally different from deduction games which traditionally have small hypothesis spaces (Clue - 324, Mystery of the Abbey - 24, Mastermind - I've seen 6561) with each hypothesis being initially equiprobable.
I've seen variants that can be played online with letters or numbers instead of pyramids, but frankly they're not nearly as fun.
"As of now, at least, reasoning based on scanty evidence is something that modern-day science cannot reliably train modern-day scientists to do at all."
By definition, scientists must use induction. Meant to say thinkers. IDK why thinkers mostly use induction now: maybe because the scientific funding model seems to work okay or because once you induce too far ahead, the content becomes useless if new research deviates the course a bit. For instance, all GUT/TOE physicists use Einstein-ian deduction in their elegant models. Einstein was lucky to be redeemed so quickly in that novel observatories were just being constructed. It is more expensive (maybe risky too) to turn the galaxy into a giant particle accelerator. In social sciences fileds, there is deduction. M.Yunus stimulated microfinance with a $26? loan by deducing collateral isn't a primary motivator in debt repayment (primary are entrepreneurial drive and quality-of-living gains). Drexler's nanotechnology vision was deduction. Many political programmes are deductions.
I agree with the general body content deduction is underappreciated. On reflection, the reason may be because an act of deduction almost always occurs in fields where there is no competing induction (ie. R.Freitas's simulations probably render much of E.Drexler's deductions obsolete). Thus deduction is a proxy to unearth low-hanging fruit? Deductive GUTs are fine, but will certainly be eclipsed by induced particle accelerator engineering blueprints one day. Deduction is free and addresses the issue of hypothesis generation somewhat.
I disagree strongly with the suggestion Einstein was a proponent of MWI. In fact, the overemphasis on deduction (defined here as induction from few au priors) caused him to waste the remaining 2/3 of his life attempting to disprove quantum phenomena, no?
Hopefully, ignoring ethics, cloning people for whatever reason will only ensure one of three (even less considering genetic mutations) character traits for whatever Eugenics you are practising. There is nurture and there is personal inspiration (probably could be defined here as intensity of rationality). If there is no Earth Summit in 1992, I probably don't pick up a bunch of environmental pamphlets one weekend, then. My decade-later clone exposed to Fox News maybe even exacerbates the leading extinction threat. Maybe if I don't grow up with cats, I don't make the inspired choice to value living beings; maybe my Fox News clone values killing Muslims and other "infidels" instead? If Eliezer doesn't read whichever sci-fi story inspired him, does he make the choice to focus upon AGI?
Spellchecker, fixed.
The Uncredible Hallq, I thought Feynman's intro to SR in the Lectures was perfectly okay. And I think John Baez has up an intro to GR.
But mostly, I haven't seen textbooks botching the explanation, because relativity is not something that you are supposed to be or allowed to be confused by, and so there is no excuse for confusing students. And because no one is "interpreting" the equations, of course. Three cheers for physical realism!
Re evolutionary psychology material, I strongly recommend The Moral Animal and Human Evolutionary Psychology to all fellow travelers.
In this post and the last you appear to be taking the opposite tack from the position you held in the discussion with Tom McCabe attached to Einstein's Arrogance. For example, you seemed to react poorly to the idea that the Einstein field equation has a relatively small information content, but later suggested that an AGI might get to that equation by watching an apple fall. Is this a shift in your position, or is there a distinction I've missed?
"IIRC, Einstein wasn't the first to try to develop a curvature theory of gravity. Riemann himself apparently tried. And, IIRC, Einstein was one of Riemann's students. Einstein brought to the table the whole thing about having to deal with spacetime rather than space."
Riemann died in 1866, Einstein was born in 1879. Riemann was a mathematician: he developed the math of differential geometry, among a great deal of other things, so a lot of stuff is named after him. Einstein applied Riemann's geometry to the physical universe. So far as I know, none of the early non-Euclidean geometry people thought that their geometries might be applicable in reality. The first theorems of hyperbolic geometry were produced in an attempt to create a contradiction and so prove Euclid's fifth postulate.
"I disagree strongly with the suggestion Einstein was a proponent of MWI. In fact, the overemphasis on deduction (defined here as induction from few au priors) caused him to waste the remaining 2/3 of his life attempting to disprove quantum phenomena, no?"
I have to find an actual physicist to discuss this with, but there appears to be nothing wrong with Einstein's quest for a unified theory; he simply didn't have the prerequisite information of QM at the time (Feynman, Dyson, etc. didn't develop renormalization until the 1940s). MWI wasn't proposed until several years after Einstein's death.
"A willingness to reconsider his assumptions, an openness to new explanations, and an abiding belief that hypotheses should always be tested against the data - and discarded if they were found wanting."
Plenty of scientists have these, and many of them make significant discoveries in their fields. But what was it about Einstein that let him discover, not one, but two of the fundamental theories of physics?
Celeriac, the distinction is that Tom McCabe seemed to me to be suggesting that the search space was small to begin with - rather than realizing the work it took to cut the search space itself down.
"Celeriac, the distinction is that Tom McCabe seemed to me to be suggesting that the search space was small to begin with - rather than realizing the work it took to cut the search space itself down."
The search space, within differential geometry, was fairly small by Einstein's day. It was a great deal of work to narrow the search space, but most of it was done by others (Conservation of Energy, various mathematical theorems, etc., were all known in 1910). The primary difficulties were in realizing that space could be described by differential geometry, and then in deriving GR from known postulates. Neither of these involve large search spaces; the former follows quickly once you realize that your assumptions are inconsistent with Minkowski space, and there's only one possible derivation of GR if you do the math correctly. I don't know why the first one is hard, but Einstein showed twice that physicists are very reluctant to question background assumptions (linear time for SR, Euclidean space for GR), so we know it must be. The second one is hard because the human brain does not come equipped with a differential geometry lobe- it took me several hours to fully understand the derivation of the Schwarzschild solution from its postulates, even though the math is simple by GR standards and there is only one possible answer (see http://en.wikipedia.org/wiki/Deriving_the_Schwarzschild_solution).
Oh it took you several hours, did it?
It took me about a year to get through The Moral Animal, reading slowly and reflecting on everything. And The Moral Animal is an introduction to the subject, with no math that I can recall except what is required to explain the Prisoner's Dilemma.
"I have to find an actual physicist to discuss this with, but there appears to be nothing wrong with Einstein's quest for a unified theory; he simply didn't have the prerequisite information of QM at the time (Feynman, Dyson, etc. didn't develop renormalization until the 1940s). MWI wasn't proposed until several years after Einstein's death."
I can't recall what renormalization is. I think there is something wrong with Einstein's quest; he was akin to Aristotle's atom theory. The Sung Dynasty was about the earliest atoms could be empirically uncovered, and a GUT is about as far away from Einstein in terms of knowledge base. I actually think Einstein's biggest accomplishment was political: writing to FDR about the possibility of a nuke. Einstein is responsible in this regard for a year of robotics, car, and computer progress along with tens of millions of present Japanese and American lives. I think the two characteristics that allowed Einstein to make 3 huge discoveries (Brownian motion, SR, GR) were his rich family that got him his patent clerk job and his willingness to be aloof and not follow the Popper-ian knowledge base of the time. I doubt he was the first to notice something wrong with phlogistan, but no one had the spare time and the determination to retool the knowledge base from ground zero (has anyone else ever taken an eight year diversion into mathematics to solve a single physics problem?). I don't think he had the same respect for quantum theory, despite founding it, that he did for GR. It seemed like he was trying to graft "quantum effects that functioned as non-local wormholes" onto GR, rather than genuinely finding a GUT by respecting quantum theory. No doubt he would have immediately championed MWI, but it seems like he was genuinely trying to undercut Copenhagen Interpretation rather than building upon it (this is in response to EY's MWI comment in the thread starter). All I'm saying is that if he would've realized the limits of his deductive method, he might've made even more contributions in his latter years and been the greatest thinker ever, instead of sharing the mantle with a handful of others. Maybe the most cutting edge scientific field is genetics. Someone might be able to deduce a science of the behaviour of animal-human hybrids studying the input animal temperaments and physiologies, but a better avenue would be to be a protein folding scientist and learn how to cure cancer or diabetes or something. I don't want to speak for Einstein's study strengths and weaknesses, but maybe we'd have optical computers now if Einstein would've transitioned to optics instead of lasers. I can't think of any physical knowledge areas now that are in as bad shape as cosmology was pre-Einstein. The next Einstein will come from social science fields, probably (is why I mentioned M.Yunus). With computers, everything physics is research teams nowadays. Maybe M.Lazaridis funding a quantum computer research park, is the closest anyone now can come to advancing a theoretical physics field as much as Einstein (cosmology) did.
Tom: yow, sorry for the inaccurate info then. I was sure I had read he was a student of Riemann's.
Either way, I thought I read that someone had been trying, and failing, to do a space curvature based theory of gravity before Einstein. But now I'm rather unsure. Oh well. Thanks for the correction.
Oh, speaking of brainfizzles on my part, may as well disregard my hunch above about relations being the "real" configuration space. The intuition that led me to that was based on a really really bad mental miscalculation of the number of degrees of freedom, confusing the dimensionality of the configuration space with the dimensionality of the associated hilbert space, and basically a whole bunch of mental booboos. So while the notion of 'violations' of some identities and inequalities might fit with curvature is still interesting to me, I'm, well, unbetting on that model. eye eeeeesh shtuuupid. :)
When Einstein invented General Relativity, he had almost no experimental data to go on, except the precession of Mercury's perihelion. And (AFAIK) Einstein did not use that data, except at the end.
Eliezer, I'd love to believe that too, but from the accounts I've read I don't think it's quite right. Because of his "hole argument", Einstein took a long detour from the correct path in 1913-1915. During that time, he abandoned his principle of general covariance, and tried to find field equations that would "work well enough in practice anyway." Apparently, one of the main reasons he finally abandoned that line of thought, and returned to general covariance, is that he was getting a prediction for Mercury's perihelion motion that was too small by a factor of 2.
So is it possible that not even Einstein was a Bayesian superintelligence?
Richard:It took me about a year to get through The Moral Animal
What was it you think slowed you down? I got through it fairly quickly & I'm pretty sure I'm not smarter than you.
I'm genuinely curious - I was very slow with books of knowledge (if I finished them at all) till about last summer, the problem (among others) fixed itself, and the question of why (or rather, how) is driving me mad.
Einstein was able to arrive at all that because he submitted his own thinking to serious constraints. He never invented new things (multilapse theory worldpretation) but actually destroyed them.
What I don't buy from his arguments though is that somehow gravitational waves would accelerate you. The universe is already accelerating, there is only the need for the waves to appear to main relativity. They don't need to be the cause of acceleration, only if you assume the universe is not accelerating.
Of course there's also the question of other reference points like looking at the stars and how they behave, but that would be too anti-dialectical for the last 300 years of philosophysical thought.
I think the post here gives the impression that Einstein made fewer errors, and had fewer detours than he did.
It is true that there were very few degrees of freedom in GR. When the initial red shift numbers came in, they seemed to contradict GR. Einstein was pondering abandoning it, until better numbers came in. There was nothing he could tweak to adjust the answer.
Also, he made the wrong prediction for the bending of starlight around the sun due to missing something (a second order effect) in the calculation. Through several strokes of luck, all the expeditions to check the values were stymied for one reason or another during the time Einstein had the wrong numbers out there. It was only after Einstein realized his mistake and redid the calculations that an expedition finally succeeded in getting the measurements which were right. Had an earlier expedition succeeded, Einstein's prediction at that time would have been wrong.
Einstein did use a lot of data though it was data everyone else had too. He knew Newton's law was mostly accurate, a fact which implies a lot of data. Also the invariance of gravitational acceleration (gravitational mass = inertial mass). He was aware there was a problem with Mercury. Also Newton's law was not (or only with great difficulty) consistent with SR.
There is a very interesting book "General relativity conflict and rivalries : Einstein's polemics with physicists" by Galina Weinstein.
Einstein was incredible but not alien magic. An algebra mistake cost him a couple of years!
Great observation. One inaccuracy is that velocity in special relativity isn't quite the same as acceleration in GR - since we can actually locally measure acceleration, and therefore know if we're accelerating or the rest of the universe is. This is unless you also count spacetime itself in the rest of the universe, in which case it's best to specify it or avoid the issue more decisively. The actual equivalence is accelerating vs. staying in constant velocity/still in a gravitational field.
Another interesting point is that this chain of "character of law" reasoning in the absence of experimental possibilities is the MO of the field of theoretical high energy physics, and many scientists are trained on ways to make progress anyway under these conditions. Most aren't doing as well as Einstein, but arguably things have gotten much more difficult to reason through at these levels of physics.
Yesterday I argued that the Powers Beyond Science are actually a standard and necessary part of the social process of science. In particular, scientists must call upon their powers of individual rationality to decide what ideas to test, in advance of the sort of definite experiments that Science demands to bless an idea as confirmed. The ideal of Science does not try to specify this process—we don't suppose that any public authority knows how individual scientists should think—but this doesn't mean the process is unimportant.
A readily understandable, non-disturbing example:
A scientist identifies a strong mathematical regularity in the cumulative data of previous experiments. But the corresponding hypothesis has not yet made and confirmed a novel experimental prediction—which his academic field demands; this is one of those fields where you can perform controlled experiments without too much trouble. Thus the individual scientist has readily understandable, rational reasons to believe (though not with probability 1) something not yet blessed by Science as public knowledge of humankind.
Noticing a regularity in a huge mass of experimental data, doesn't seem all that unscientific. You're still data-driven, right?
But that's because I deliberately chose a non-disturbing example. When Einstein invented General Relativity, he had almost no experimental data to go on, except the precession of Mercury's perihelion. And (AFAIK) Einstein did not use that data, except at the end.
Einstein generated the theory of Special Relativity using Mach's Principle, which is the physicist's version of the Generalized Anti-Zombie Principle. You begin by saying, "It doesn't seem reasonable to me that you could tell, in an enclosed room, how fast you and the room were going. Since this number shouldn't ought to be observable, it shouldn't ought to exist in any meaningful sense." You then observe that Maxwell's Equations invoke a seemingly absolute speed of propagation, c, commonly referred to as "the speed of light" (though the quantum equations show it is the propagation speed of all fundamental waves). So you reformulate your physics in such fashion that the absolute speed of a single object no longer meaningfully exists, and only relative speeds exist. I am skipping over quite a bit here, obviously, but there are many excellent introductions to relativity—it is not like the horrible situation in quantum physics.
Einstein, having successfully done away with the notion of your absolute speed inside an enclosed room, then set out to do away with the notion of your absolute acceleration inside an enclosed room. It seemed to Einstein that there shouldn't ought to be a way to differentiate, in an enclosed room, between the room accelerating northward while the rest of the universe stayed still, versus the rest of the universe accelerating southward while the room stayed still. If the rest of the universe accelerated, it would produce gravitational waves that would accelerate you. Moving matter, then, should produce gravitational waves.
And because inertial mass and gravitational mass were always exactly equivalent—unlike the situation in electromagnetics, where an electron and a muon can have different masses but the same electrical charge—gravity should reveal itself as a kind of inertia. The Earth should go around the Sun in some equivalent of a "straight line". This requires spacetime in the vicinity of the Sun to be curved, so that if you drew a graph of the Earth's orbit around the Sun, the line on the 4D graph paper would be locally flat. Then inertial and gravitational mass would be necessarily equivalent, not just coincidentally equivalent.
(If that did not make any sense to you, there are good introductions to General Relativity available as well.)
And of course the new theory had to obey Special Relativity, and conserve energy, and conserve momentum, etcetera.
Einstein spent several years grasping the necessary mathematics to describe curved metrics of spacetime. Then he wrote down the simplest theory that had the properties Einstein thought it ought to have—including properties no one had ever observed, but that Einstein thought fit in well with the character of other physical laws. Then Einstein cranked a bit, and got the previously unexplained precession of Mercury right back out.
How impressive was this?
Well, let's put it this way. In some small fraction of alternate Earths proceeding from 1800—perhaps even a sizeable fraction—it would seem plausible that relativistic physics could have proceeded in a similar fashion to our own great fiasco with quantum physics.
We can imagine that Lorentz's original "interpretation" of the Lorentz contraction, as a physical distortion caused by movement with respect to the ether, prevailed. We can imagine that various corrective factors, themselves unexplained, were added on to Newtonian gravitational mechanics to explain the precession of Mercury—attributed, perhaps, to strange distortions of the ether, as in the Lorentz contraction. Through the decades, further corrective factors would be added on to account for other astronomical observations. Sufficiently precise atomic clocks, in airplanes, would reveal that time ran a little faster than expected at higher altitudes (time runs slower in more intense gravitational fields, but they wouldn't know that) and more corrective "ethereal factors" would be invented.
Until, finally, the many different empirically determined "corrective factors" were unified into the simple equations of General Relativity.
And the people in that alternate Earth would say, "The final equation was simple, but there was no way you could possibly know to arrive at that answer from just the perihelion precession of Mercury. It takes many, many additional experiments. You must have measured time running slower in a stronger gravitational field; you must have measured light bending around stars. Only then can you imagine our unified theory of ethereal gravitation. No, not even a perfect Bayesian superintelligence could know it!—for there would be many ad-hoc theories consistent with the perihelion precession alone."
In our world, Einstein didn't even use the perihelion precession of Mercury, except for verification of his answer produced by other means. Einstein sat down in his armchair, and thought about how he would have designed the universe, to look the way he thought a universe should look—for example, that you shouldn't ought to be able to distinguish yourself accelerating in one direction, from the rest of the universe accelerating in the other direction.
And Einstein executed the whole long (multi-year!) chain of armchair reasoning, without making any mistakes that would have required further experimental evidence to pull him back on track.
Even Jeffreyssai would be grudgingly impressed. Though he would still ding Einstein a point or two for the cosmological constant. (I don't ding Einstein for the cosmological constant because it later turned out to be real. I try to avoid criticizing people on occasions where they are right.)
What would be the probability-theoretic perspective on Einstein's feat?
Rather than observe the planets, and infer what laws might cover their gravitation, Einstein was observing the other laws of physics, and inferring what new law might follow the same pattern. Einstein wasn't finding an equation that covered the motion of gravitational bodies. Einstein was finding a character-of-physical-law that covered previously observed equations, and that he could crank to predict the next equation that would be observed.
Nobody knows where the laws of physics come from, but Einstein's success with General Relativity shows that their common character is strong enough to predict the correct form of one law from having observed other laws, without necessarily needing to observe the precise effects of the law.
(In a general sense, of course, Einstein did know by observation that things fell down; but he did not get GR by backward inference from Mercury's exact perihelion advance.)
So, from a Bayesian perspective, what Einstein did is still induction, and still covered by the notion of a simple prior (Occam prior) that gets updated by new evidence. It's just the prior was over the possible characters of physical law, and observing other physical laws let Einstein update his model of the character of physical law, which he then used to predict a particular law of gravitation.
If you didn't have the concept of a "character of physical law", what Einstein did would look like magic—plucking the correct model of gravitation out of the space of all possible equations, with vastly insufficient evidence. But Einstein, by looking at other laws, cut down the space of possibilities for the next law. He learned the alphabet in which physics was written, constraints to govern his answer. Not magic, but reasoning on a higher level, across a wider domain, than what a naive reasoner might conceive to be the "model space" of only this one law.
So from a probability-theoretic standpoint, Einstein was still data-driven—he just used the data he already had, more effectively. Compared to any alternate Earths that demanded huge quantities of additional data from astronomical observations and clocks on airplanes to hit them over the head with General Relativity.
There are numerous lessons we can derive from this.
I use Einstein as my example, even though it's cliche, because Einstein was also unusual in that he openly admitted to knowing things that Science hadn't confirmed. Asked what he would have done if Eddington's solar eclipse observation had failed to confirm General Relativity, Einstein replied: "Then I would feel sorry for the good Lord. The theory is correct."
According to prevailing notions of Science, this is arrogance—you must accept the verdict of experiment, and not cling to your personal ideas.
But as I concluded in Einstein's Arrogance, Einstein doesn't come off nearly as badly from a Bayesian perspective. From a Bayesian perspective, in order to suggest General Relativity at all, in order to even think about what turned out to be the correct answer, Einstein must have had enough evidence to identify the true answer in the theory-space. It would take only a little more evidence to justify (in a Bayesian sense) being nearly certain of the theory. And it was unlikely that Einstein only had exactly enough evidence to bring the hypothesis all the way up to his attention.
Any accusation of arrogance would have to center around the question, "But Einstein, how did you know you had reasoned correctly?"—to which I can only say: Do not criticize people when they turn out to be right! Wait for an occasion where they are wrong! Otherwise you are missing the chance to see when someone is thinking smarter than you—for you criticize them whenever they depart from a preferred ritual of cognition.
Or consider the famous exchange between Einstein and Niels Bohr on quantum theory—at a time when the then-current, single-world quantum theory seemed to be immensely well-confirmed experimentally; a time when, by the standards of Science, the current (deranged) quantum theory had simply won.
Einstein: "God does not play dice with the universe."
Bohr: "Einstein, don't tell God what to do."
You've got to admire someone who can get into an argument with God and win.
If you take off your Bayesian goggles, and look at Einstein in terms of what he actually did all day, then the guy was sitting around studying math and thinking about how he would design the universe, rather than running out and looking at things to gather more data. What Einstein did, successfully, is exactly the sort of high-minded feat of sheer intellect that Aristotle thought he could do, but couldn't. Not from a probability-theoretic stance, mind you, but from the viewpoint of what they did all day long.
Science does not trust scientists to do this, which is why General Relativity was not blessed as the public knowledge of humanity until after it had made and verified a novel experimental prediction—having to do with the bending of light in a solar eclipse. (It later turned out that particular measurement was not precise enough to verify reliably, and had favored GR essentially by luck.)
However, just because Science does not trust scientists to do something, does not mean it is impossible.
But a word of caution here: The reason why history books sometimes record the names of scientists who thought great high-minded thoughts, is not that high-minded thinking is easier, or more reliable. It is a priority bias: Some scientist who successfully reasoned from the smallest amount of experimental evidence got to the truth first. This cannot be a matter of pure random chance: The theory space is too large, and Einstein won several times in a row. But out of all the scientists who tried to unravel a puzzle, or who would have eventually succeeded given enough evidence, history passes down to us the names of the scientists who successfully got there first. Bear that in mind, when you are trying to derive lessons about how to reason prudently.
In everyday life, you want every scrap of evidence you can get. Do not rely on being able to successfully think high-minded thoughts unless experimentation is so costly or dangerous that you have no other choice.
But sometimes experiments are costly, and sometimes we prefer to get there first... so you might consider trying to train yourself in reasoning on scanty evidence, preferably in cases where you will later find out if you were right or wrong. Trying to beat low-capitalization prediction markets might make for good training in this?—though that is only speculation.
As of now, at least, reasoning based on scanty evidence is something that modern-day science cannot reliably train modern-day scientists to do at all. Which may perhaps have something to do with, oh, I don't know, not even trying?
Actually, I take that back. The most sane thinking I have seen in any scientific field comes from the field of evolutionary psychology, possibly because they understand self-deception, but also perhaps because they often (1) have to reason from scanty evidence and (2) do later find out if they were right or wrong. I recommend to all aspiring rationalists that they study evolutionary psychology simply to get a glimpse of what careful reasoning looks like. See particularly Tooby and Cosmides's "The Psychological Foundations of Culture".
As for the possibility that only Einstein could do what Einstein did... that it took superpowers beyond the reach of ordinary mortals... here we run into some biases that would take a separate post to analyze. Let me put it this way: It is possible, perhaps, that only a genius could have done Einstein's actual historical work. But potential geniuses, in terms of raw intelligence, are probably far more common than historical superachievers. To put a random number on it, I doubt that anything more than one-in-a-million g-factor is required to be a potential world-class genius, implying at least six thousand potential Einsteins running around today. And as for everyone else, I see no reason why they should not aspire to use efficiently the evidence that they have.
But my final moral is that the frontier where the individual scientist rationally knows something that Science has not yet confirmed, is not always some innocently data-driven matter of spotting a strong regularity in a mountain of experiments. Sometimes the scientist gets there by thinking great high-minded thoughts that Science does not trust you to think.
I will not say, "Don't try this at home." I will say, "Don't think this is easy." We are not discussing, here, the victory of casual opinions over professional scientists. We are discussing the sometime historical victories of one kind of professional effort over another. Never forget all the famous historical cases where attempted armchair reasoning lost.