= 27d567bc2d48d9f40e9ccc20ea370b15)
Eliezer_Yudkowsky comments on Common sense as a prior - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (212)
Did you happen to read (perhaps an abbreviated version of) the QM sequence on LW, e.g. this one?
Of course I would stake my reply most strongly on 2 (single-world QM simply doesn't work) with a moderate dose of 1 (great physicists may be bad epistemologists and not know about Solomonoff Induction, formal definitions of simplicity in Occam's Razor, or how to give up and say oops, e.g. many may be religious which sets very harsh upper bounds on how much real discipline their subject could systematically teach on reductionist epistemology, rejection of complex inadequately supported privileged hypotheses, and saying oops when nobody is holding a gun to your head, yes this is a fair critique). And with that said, I reject the question 3 as being profoundly unhelpful. It's evident from history that the state of affairs postulated in 1 and 2 is not improbable enough to require some vastly difficult thesis about inhumanly superior rationality! I don't need a hero license!
This would serve as one of my flagship replies to Carl's question with respect to that portion of the audience which is capable of putting their metaness on hold long enough to see that single-world QM has negligible probability on the object level. Unfortunately, majoritarianism is a closed system in terms of rejecting all evidence against itself, when you take the 'correct' answer for comparison purposes to be the majoritarian one.
I haven't read the QM sequence. The marginal value of reading it (given its length) seemed too low to give it priority over other things, but I'm open to reconsidering. My comments above and here are entirely outside view in nature.
It could be that one can reformulate QM in an entirely different language that makes it clear that some version of single-world QM does work. Obviously you have more subject matter knowledge than I do, but I know of examples from math where an apparently incoherent mathematical concepts turned out to be rigorously formalizable. (The Dirac delta-function is perhaps an example.)
It could be that your analysis is confused. As far as I know, it hasn't been vetted by many people with subject matter knowledge, and analysis that hasn't been vetted often turns out to be wrong. Confidence in the correctness of one's reasoning at the 99+% level is really high.
There could be equally strong arguments against many worlds. My understanding is that the broad consensus among physicists is that no interpretation is satisfactory.
While Solomonoff Induction and formalized definitions of simplicity are in some sense relevant, it's not clear that they're necessary to get the right answer here.
Epistemic soundness isn't a univariate quantity
I find it hard to imagine how you could have gotten enough evidence about the rationality of physicists to warrant 99+% confidence.
According to Wikipedia, Bell, Bethe, Bohr, Dirac, Everett Feynman, Higgs, Oppenheimer, Penrose, Schrödinger and Wigner were all atheists.
Have you had enough exposure to these people to be able to make an assessment of this type?
Great physicists are in substantial part motivated by intellectual curiosity, which corresponds to a desire to get things having to do with physics right.
What history?
Not improbable at the 99+% level?
I don't believe in majoritarianism in general. But I think that one needs an awful lot of evidence to develop very high confidence that one's views on a given subject are right when other people who have thought about it a lot disagree. I think that there's a necessity not only for object level arguments (which may be wrong in unseen ways) but also a robust base of arguments for why other people have gone astray.
Just FYI, I think Eliezer's mental response to most of the questions/responses you raised here will be "I spent 40+ pages addressing these issues in my QM sequence. I don't have time to repeat those points all over again."
It might indeed be worth your time to read the QM sequence, so that you can have one detailed, thoroughly examined example of how one can acquire high confidence that the plurality of scientific elites (even in a high-quality field like physics) are just wrong. Or, if you read the sequence and come to a different conclusion, that would also be interesting.
Even if I read the QM sequence and find the arguments compelling, I still wouldn't feel as though I had enough subject matter expertise to rationally disagree with elite physicists with high confidence. I don't think that I'm more rational than Bohr, Heisenberg, Dirac, Feynman, Penrose, Schrödinger and Wigner. These people thought about quantum mechanics for decades. I wouldn't be able to catch up in a week. The probability that I'd be missing fundamental and relevant things that they knew would dominate my prior.
I'll think about reading the QM sequence.
The problem is, he haven't spent 1..2 pages actually writing out a more-or-less formal argument as of how exactly he's comparing the complexity of two interpretations that output different sorts of data. Attempts at production of such arguments tend to de-confuse people who think they see an answer.
There's 40 pages of, mostly, second-order popularization of some basics of QM (if the author has only learned QM from popularizations himself and haven't ensured correctness with homework), rife with things like incitement to feel better about yourself if you agree with the author, allusions to how the author can allegedly write the interpretations as TM programs, the author 'explaining' why physicists don't agree with him, and so on.
Ultimately, any worthwhile conclusion about MWI is to an extreme extent dependent on extensive domain specific expertise in physics. One could perhaps argue that it also requires rationality, but one can't argue that one doesn't need to know physics. It doesn't really matter to physicists what's the most rational thing to believe when you lack said knowledge, just as Monty in Monty Hall who knows where the goat is doesn't care much about the highest probability door given ignorance of where the goat is.
We should separate "rationality" from "domain knowledge of quantum physics."
Certainly, each of these figures had greater domain knowledge of quantum physics than I plan to acquire in my life. But that doesn't mean I'm powerless to (in some cases) tell when they're just wrong. The same goes for those with immense domain knowledge of – to take an unusually clear-cut example – Muslim theology.
Consider a case where you talk to a top Muslim philosopher and extract his top 10 reasons for believing in Allah. His arguments have obvious crippling failures, so you try to steel man them, and you check whether you're just misinterpreting the arguments, but in the end they're just... terrible arguments. And then you check with lots of other leading Muslim philosophers and say "Give me your best reasons" and you just get nothing good. At that point, I think you're in a pretty good position to reject the Muslim belief in Allah, rather than saying "Well, maybe there's some subtle point they know about that I haven't gotten to in the first 40 hours of investigation. They are the experts on Allah, so I'm probably not in a good position to confidently disagree with them."
The case with QM is obviously different in many ways, but the same point that it's possible to (rationally) confidently disagree with the plurality or majority of experts on something still stands, I think.
One of the biggest differences between Muslim theology and QM is that the QM experts seem to have much better rationality than the Muslim theologians, so it's not as easy to conclude that you're evaluating the evidence and arguments more appropriately than they are.
And this brings us to what might be a significant difference between us. I tend to think it's quite feasible to surpass the rationality skill of (most) top-performing physicists. Perhaps you think that is not that feasible.
Let me make another analogy. Consider the skill of nonfiction writing. Rousseau and Hume were great for their time, but Pinker and Dawkins write better because they've got enormous advantages: they get to study Rousseau and Hume, they get to study later writers, they have instant access to online writing-helpers like Thesaurus.com and reading-level measures, they get to benefit from hundreds of years of scientific study of psychology and communication, they get to email drafts to readers around the world for feedback, etc.
Similarly, I think it's quite feasible to outperform (in rationality) most of the top-performing physicists. I'm not sure that I have reached that level yet, but I think I certainly could.
Here, the enormous advantages available to me don't come from the fact that I have access to things that the scientist's don't have access to, but instead from the fact that the world's best physicists mostly won't choose to take advantage of the available resources – which is part of why they so often say silly things as soon as they step outside the lab.
Many qualifications, and specific examples, could be given, but I'll pause here and give you a chance to respond.
I feel the main disanalogy with the Muslim theology case is that elite common sense does not regard top Muslim philosophers as having much comparative expertise on the question of whether Allah exists, but they would regard top physicists as having very strong comparative expertise on the interpretation of quantum mechanics. By this I mean that elite common sense would generally substantially defer to the opinions of the physicists but not the Muslim philosophers. This disanalogy is sufficiently important to me that I find the overall argument by analogy highly non-compelling.
I note that there are some meaningful respects in which elite common sense would regard the Muslim philosophers as epistemic authorities. They would recognize their authority as people who know about what the famous arguments for Allah's existence and nature are, what the famous objections and replies are, and what has been said about intricacies of related metaphysical questions, for example.
The big difference is that the atheist arguments are not reliant upon Muslim philosophy in the way in which all possible MWI arguments rely on physics. The relevant questions are subtle aspects of physics - linearity, quantum gravity, etc. related, there are unresolved issues in physics to which many potential solutions are MWI unfriendly - precisely in the ballpark of your "quantum gravity vs. string theory" where you do not even know how two terms relate to each other, nor know how advanced physics relates to MWI.
1.3 billion Muslims -- including many national leaders, scientists, and other impressive people throughout the Islamic world -- believe that Mohammad was an authority on whether Allah exists.
Of course, there are plenty of others who do not, so I'm not suggesting we conclude it's probable that he was, but it seems like something we couldn't rule out, either.
Religion is a set of memes, not in the Internet sense of "catchy saying' but in the slightly older sense that it has characteristics which lead to it spreading. I suggest modifying the original proposal: in deciding who is trustworthy to many people, you should take into account that beliefs which are good at spreading for reasons unrelated to their truth value can skew the "many people" part.
Given what we know about how religions spread, religious beliefs should be excluded on these grounds alone. If scientists who expressed a particular belief about science were killed, that would apply to scientists too (the fact that most trusted scientists in Stalinist Russia believed in Lysenkoism, since the others were dead or silenced, would not be a reason for you to do so).
So is culture. Are you ready to demand culture-neutrality?
Would that apply also to scientists who were prevented from getting grants and being published? How do you know, without hindsight, which of the two warring scientific factions consists of cranks and crooks, and which one does not?
It is possible for a culture to at least not be inimical to truth. But to the extent that a religion is not inimical to truth, it has ceased to be a religion.
If the main reason why scientists don't express a belief is that if they do they would be arbitrarily denied grants and publication, then it would apply. However, in the modern Western world, almost every example* where someone made this claim has turned out to be a crank whose lack of publication was for very good reasons. As such, my default assumption will be that this has not occurred unless I have a specific reason to believe that it has.
If by rationality we mean more accurate and correct reasoning... it so happens that in the case of existence of many worlds, any rational conclusion is highly dependent on the intricacies of physics (specifically related to the quantum field theory and various attempts at quantum gravity), and consequently, conclusions being reached without such knowledge is a sign of very poor rationality just as equation of a+b*7+c = 0 having been simplified to a=1 without knowing anything about the values of b and c is a sign of very poor knowledge of mathematics, not of some superior equation simplification skill.
(The existence of Allah, God, Santa Claus, or what ever is, of course, not likewise dependent on theology).
My impression was that the conclusion in fact just depends on one's interpretation of Occam's razor, rather than the intricacies of physics. I had allowed myself to reach a fairly tentative conclusion about Many Worlds because physicists seem to agree that both are equally consistent with the data. We are then left with the question of what 'the simplest explanation' means, which is an epistemology question, not a physics question, and one I feel comfortable answering.
Am I mistaken?
Yes, you are (mistaken). As numerous PhD physicists have been saying numerous times on this site and elsewhere, the issue is that QM is not consistent with observations (does not include gravity). Neither is QFT.
The question is one of fragility of MWI over potential TOEs, and, relying on exact linearity it is arguably very fragile.
Furthermore with regards to Occam's razor and specifically formalizations of it, that is also a subtle question requiring domain specific training.
In particular, in Solomonoff Induction, codes have to produce output that exactly matches the observations. Complete with the exact picture of photon noise on the double slit experiment's screen. Outputting a list of worlds like MWI does is not even an option.
Interesting. I'll assume an agnostic position again for the time being.
Can you point me towards some of the best comments?
I was aware that both theories are inconsistent with data with respect to gravity, obviously if either of them weren't, the choice would be clear.
What do you mean by 'fragility over' potential theories of everything? That the TOEs suggested thus far tend not to be compatible with it? Presumably not given that the people generating the TOEs are likely to start with the most popular theory.
Whats the standard response by MW enthusiasts to your point on Solomonof induction? My understanding would then suggest that neither MW nor Copenhagen can give an exact picture of photon noise, in which case the problem would seem to be with Solomonoff induction as a formalization.
There's some around this thread (responses to Luke's comment). Also I think that QM sequence has responses from physicists.
The MWI is concluded from exactly linear quantum mechanics. Because we know QM to be only an approximation, we lack any strong reasons to expect exact linearity in the final TOE. Furthermore even though exact linearity is arguably favoured by the Occam's razor over any purely speculative non-linear theory, that does not imply that it is more probable than all of the nonlinear theories together (which would have same linear approximation).
In my opinion, things like multitude of potential worlds allow for e.g. elegantly (and compactly) expressing some conservation laws as survivor bias (via some sort of instability destroying observers in the world where said laws do not hold). Whenever that is significant to TOEs is, of course, purely speculative.
As far as I know, the arguments that Solomonoff induction supports MWI never progressed beyond mere allusions to such support.
In raw form, yes, neither interpretation fits and it's unclear how to compare complexities of them formally.
I explored some on how S.I. would work on data from quantum experiments here. Basically, the task is to represent said photon noise with the minimum amount of code and data, which can be done in two steps by calculating probabilities as per QM and Born rule, and using the probability density function to decode photon coordinates from the subsequent input bits. (analogous to collapse), or perhaps more compactly in one step by doing QM with some sort of very clever bit manipulation on strings of random noise as to obtain desired probability distribution in the end.
You need to keep in mind that they have access to a very, very huge number of things that you don't have access to (and have potential access to more things than you think they do), and can get the taste of the enormous space of issues any of which can demolish the case for MWI that you see completely, from the outside. For example, non-linearity of the equations absolutely kills MWI. Now, only a subset of physicists considers the non-linearity to be likely. Others believe in other such things, most of which just kill MWI outright, with all the argument for it flying apart like a card house in the wind of a category 6 hurricane. Expecting a hurricane, experts choose not to bother with the question of mechanical stability of the card house on it's own - they would rather debate in which direction, and how far, the cards will land.
Thanks for the thoughtful response. I agree that there are many instances in which it’s possible to rationally come to confident conclusions that differ from those of subject matter experts. I realize that my earlier comment was elliptical, and will try to clarify. The relevant points to my mind are:
The extraordinary intellectual caliber of the best physicists
Though difficult to formalize, I think that there's a meaningful sense in which one can make statements of the general form "person A has intellectual caliber n times that of person B." Of course, this is domain specific to some degree, but I think that the concept hangs together somewhat even across domains.
One operationalization of this is "if person B reaches a correct conclusion on a given subject, person A could reach it n times as fast." Another is "it would take n copies of person B to do person A's work." These things are hard to estimate, but one can become better calibrated by using the rule "if person A has intellectual caliber n times that of person B and person B has intellectual caliber m times that of person C, then person A has intellectual caliber n*m times that of person C."
In almost all domains, I think that the highest intellectual caliber people have no more than 5x my intellectual caliber. Physics is different. From what I’ve heard, the distribution of talent in physics is similar to that of math. The best mathematicians are 100x+ my intellectual caliber. I had a particularly striking illustrative experience with Don Zagier, who pinpointed a crucial weakness in an analogy that I had been exploring for 6 months (and which I had run by a number of other mathematicians) in a mere ~15 minutes. I would not be surprised if he himself were to have an analogous experience with the historical greats.
When someone is < 5x one’s intellectual caliber, an argument of the type “this person may be smarter than me, but I’ve focused a lot more on having accurate views, so I trust my judgment over him or her” seems reasonable. But when one gets to people who are 100x+ one’s intellectual caliber, the argument becomes much weaker. Model uncertainty starts to play a major role. It could be that people who are that much more powerful easily come to the correct conclusion on a given question without even needing to put conscious effort into having accurate beliefs.
The intrinsic interest of the question of interpretation of quantum mechanics
The question of what quantum mechanics means has been considered one of the universe’s great mysteries. As such, people interested in physics have been highly motivated to understand it. So I think that the question is privileged relative to other questions that physicists would have opinions on — it’s not an arbitrary question outside of the domain of their research accomplishments.
Solicitation of arguments from those with opposing views
In the Muslim theology example, you spend 40 hours engaging with the Muslim philosophers. This seems disanalogous to the present case, in that as far as I know, Eliezer’s quantum mechanics sequence hasn’t been vetted by any leading physicists who disagree with the many world’s interpretation of quantum mechanics. I also don’t know of any public record of ~40 hours of back and forth analogous to the one that you describe. I know that Eliezer might cite an example in his QM sequence, and will take a look.
Let me first say that I find this to be an extremely interesting discussion.
I think there is a social norm in mathematics and physics that requires people to say this, but I have serious doubts about whether it is true. Anyone 100x+ your intellectual caliber should be having much, much more impact on the world (to say nothing of mathematics itself) than any of the best mathematicians seem to be having. At the very least, if there really are people of that cognitive level running around, then the rest of the world is doing an absolutely terrible job of extracting information and value from them, and they themselves must not care too much about this fact.
More plausible to me is the hypothesis that the best mathematicians are within the same 5x limit as everyone else, and that you overestimate the difficulty of performing at their level due to cultural factors which discourage systematic study of how to imitate them.
Try this thought experiment: suppose you were a graduate student in mathematics, and went to your advisor and said: "I'd like to solve [Famous Problem X], and to start, I'm going to spend two years closely examining the work of Newton, Gauss, and Wiles, and their contemporaries, to try to discern at a higher level of generality what the cognitive stumbling blocks to solving previous problems were, and how they overcame them, and distill these meta-level insights into a meta-level technique of my own which I'll then apply to [Famous Problem X]." What do you think the reaction would be? How many times do you think such a thing has ever been proposed, let alone attempted, by a serious student or (even) senior researcher?
Nice to hear from you :-)
Yes, I think that this is what the situation is.
I'll also say that I think that there are very few such people — maybe on the order of 10 who are alive today. With such a small absolute number, I don't think that their observed impact on math is a lot lower than what one would expect a priori, and the prior in favor them having had a huge impact in society isn't that strong.
"The best mathematicians are 100+x higher in intellectual caliber than I am" and "the difference is in large part due to cultural factors which discourage systematic study of how to imitate them" aren't mutually exclusive. I'm sympathetic to your position.
To change the subject :-)
Basically never.
Not to mention that some of them might be working on Wall Street or something, and not have worked on unsolved problems in mathematics in decades.
This is a terrible idea unless they're spending half their time pushing their limits on object-level math problems. I just don't think it works to try to do a meta phase before an object phase unless the process is very, very well-understood and tested already.
I'm sure that's exactly what the advisor would say (if they bother to give a reasoned reply at all), with the result that nobody ever tries this.
(I'll also note that it's somewhat odd to hear this response from someone whose entire mission in life is essentially to go meta on all of humanity's problems...)
But let me address the point, so as not to be logically rude. The person would be pushing their limits on object-level math problems in the course of "examining the work of Newton, Gauss, and Wiles", in order to understand said work; otherwise, it can hardly be said to constitute a meaningful examination. I also think it's important not to confuse meta-ness with (nontechnical) "outside views"; indeed I suspect that a lot of the thought processes of mathematical "geniuses" consist of abstracting over classes of technical concepts that aren't ordinarily abstracted over, and thus if expressed explicitly (which the geniuses may lack the patience to do) would simply look like another form of mathematics. (Others of their processes, I speculate, consist in obsessive exercising of visual/dynamic mental models of various abstractions.)
Switching back to logical rudeness, I'm not sure the meta-ness is your true rejection; I suspect what you may be really worried about is making sure there are tight feedback loops to which one's reasoning can be subjected.
That's not the kind of meta I mean. The dangerous form of meta is when you spend several years preparing to do X, supposedly becoming better at doing X, but not actually doing X, and then try to do X. E.g. college. Trying to improve at doing X while doing X is much, much wiser. I would similarly advise Effective Altruists who are not literally broke to be donating $10 every three months to something while they are trying to increase their incomes and invest in human capital; furthermore, they should not donate to the same thing two seasons in a row, so that they are also practicing the skill of repeatedly assessing which charity is most important.
"Meta" for these purposes is any daily activity which is unlike the daily activity you intend to do 'later'.
Tight feedback loops are good, but not always available. This is a separate consideration from doing meta while doing object.
The activity of understanding someone else's proofs may be unlike the activity of producing your own new math from scratch; this would be the problem.
How do you know how little intellectual caliber JonahSinick has?
looks at JonahSinick's profile
follows link to his website
skims the “About me” page
Yes, you have a point.
My understanding is that the interpretation of QM is (1) not regarded as a very central question in physics, being seen more as a "philosophy" question and being worked on to a reasonable extent by philosophers of physics and physicists who see it as a hobby horse, (2) is not something that physics expertise--having good physical intuition, strong math skills, detailed knowledge of how to apply QM on concrete problems--is as relevant for as many other questions physicists work on, and (3) is not something about which there is an extremely enormous amount to say. These are some of the main reasons I feel I can update at all from the expert distribution of physicists on this question. I would hardly update at all from physicist opinions on, say, quantum gravity vs. string theory, and I think it would basically be crazy for me to update substantially in one direction or the other if I had comparable experience on that question.
[ETA: As evidence of (1), I might point to the prevalence of the "shut up and calculate" mentality which seems to have been reasonably popular in physics for a while. I'd also point to the fact that Copenhagen is popular but really, really, really, really not good. And I feel that this last claim is not just Nick Beckstead's idiosyncratic opinion, but the opinion of every philosopher of physics I have ever spoken with about this issue.]
A minor quibble.
I believe you are using bad terminology. 'Quantum gravity' refers to any attempt to reconcile quantum mechanics and general relativity, and string theory is one such theory (as well as a theory of everything). Perhaps you are referring to loop quantum gravity, or more broadly, to any theory of quantum gravity other than string theory?
Perhaps I should have meant loop quantum gravity. I confess that I am speaking beyond my depth, and was just trying to give an example of a central dispute in current theoretical physics. That is the type of case where I would not like to lean heavily on my own perspective.
I agree if we're talking about the median theoretical physicist at a top 5 school, but when one gets further toward the top of the hierarchy, one starts to see a high density of people who are all-around intellectually curious and who explore natural questions that they come across independently of whether they're part of their official research.
I agree, but a priori I suspect that philosophers of physics and others without heavy subject matter knowledge of quantum mechanics have leaned too heavily on this. Spending one's life thinking about something can result subconscious acquisition of implicit knowledge of things that are obliquely related. People who haven't had this experience may be at a disadvantage.
I actually think that it's possible for somebody without subject matter knowledge to rationally develop priors that are substantially different from expert consensus here. One can do this by consulting physicists who visibly have high epistemic rationality outside of physics, by examining sociological factors that may have led to the status quo, and by watching physicists who disagree debate each other and see which of the points they respond to and which ones they don't.
Can you give a reference?
But note that philosophers of physics sometimes make whole careers thinking about this, and they are among the most high-caliber philosophers. They may be at an advantage in terms of this criterion.
I can't think of a reference in print for my claim about what almost all philosophers think. I think a lot of them would find it too obvious to say, and wouldn't bother to write a paper about it. But, for what it's worth, I attended a couple of conferences on philosophy of physics held at Rutgers, with many leading people in the field, and talked about this question and never heard anyone express an opposing opinion. And I was taught about interpretations of QM from some leading people in philosophy of physics.
What I'm anchoring on here is the situation in the field of philosophy of math, where lack of experience with the practice of math seriously undercuts most philosophers' ability to do it well. There are exceptions, for example I consider Imre Lakatos to be one. Maybe the situation is different in philosophy of physics.
That is of course exactly why I picked QM and MWI to make my case for nihil supernum. It wouldn't serve to break a smart person's trust in a sane world if I demonstrated the insanity of Muslim theologians or politicians; they would just say, "But surely we should still trust in elite physicists." It is by demonstrating that trust in a sane world fails even at the strongest point which 'elite common sense' would expect to find, that I would hope to actually break someone's emotional trust, and cause them to just give up.
I haven't fully put together my thoughts on this, but it seems like a bad test to "break someone's trust in a sane world" for a number of reasons:
this is a case where all the views are pretty much empirically indistinguishable, so it isn't an area where physicists really care all that much
since the views are empirically indistinguishable, it is probably a low-stakes question, so the argument doesn't transfer well to breaking our trust in a sane world in high-stakes cases; it makes sense to assume people would apply more rationality in cases where more rationality pays off
as I said in another comment, MWI seems like a case where physics expertise is not really what matters, so this doesn't really show that the scientific method as applied by physicists is broken; it seems it at most it shows that physics aren't good at questions that are essentially philosophical; it would be much more persuasive if you showed that e.g., quantum gravity was obviously better than string theory and only 18% of physicists working in the relevant area thought so
[Edited to add a missing "not"]
From my perspective, the main point is that if you'd expect AI elites to handle FAI competently, you would expect physics elites to handle MWI competently - the risk factors in the former case are even greater. Requires some philosophical reasoning? Check. Reality does not immediately call you out on being wrong? Check. The AI problem is harder than MWI and it has additional risk factors on top of that, like losing your chance at tenure if you decide that your research actually needs to slow down. Any elite incompetence beyond the demonstrated level in MWI doesn't really matter much to me, since we're already way under the 'pass' threshold for FAI.
I feel this doesn't address the "low stakes" issues I brought up, or that this may not even by the physicists' area of competence. Maybe you'd get a different outcome if the fate of the world depended on this issue, as you believe it does with AI.
I also wonder if this analysis leads to wrong historical predictions. E.g., why doesn't this reasoning suggest that the US government would totally botch the constitution? That requires philosophical reasoning and reality doesn't immediately call you out on being wrong. And the people setting things up don't have incentives totally properly aligned. Setting up a decent system of government strikes me as more challenging than the MWI problem in many respects.
How much weight do you actually put on this line of argument? Would you change your mind about anything practical if you found out you were wrong about MWI?
You meant “is not really”?
Yes, thank you for catching.
I agree that if it were true that the consensus of elite physicists believed that MWI is wrong when there was a decisive case in favor of it, that would be striking. But
There doesn't seem to be a consensus among elite physicists that MWI is wrong.
Paging through your QM sequence, it doesn't look as though you've systematically addressed all objections that otherwise credible people have raised against MWI. For example, have you been through all of the references that critique MWI cited in this paper? I think given that most experts don't view the matter as decided, and given the intellectual caliber of the experts, in order have 99+% confidence in this setting, one has to cover all of one's bases.
One will generally find that correct controversial ideas convince some physicists. There are many physicists who believe MWI (though they perhaps cannot get away with advocating it as rudely as I do), there are physicists signed up for cryonics, there were physicists advocating for Drexler's molecular nanotechnology before it was cool, and I strongly expect that some physicists who read econblogs have by now started advocating market monetarism (if not I would update against MM). A good new idea should have some physicists in favor of it, and if not it is a warning sign. (Though the endorsement of some physicists is not a proof, obviously many bad ideas can convince a few physicists too.) If I could not convince any physicists of my views on FAI, that would be a grave warning sign indeed. (I'm pretty sure some such already exist.) But that a majority of physicists do not yet believe in MWI does not say very much one way or another.
The cognitive elites do exist and some of them are physicists, therefore you should be able to convince some physicists. But the cognitive elites don't correspond to a majority of MIT professors or anything like that, so you shouldn't be able to convince a majority of that particular group. A world which knew what its own elite truthfinders looked like would be a very different world from this one.
Ok, putting aside MWI, maybe our positions are significantly more similar than it initially seemed. I agree with
I've taken your comments such as
to carry connotations of the type "the fraction of people who exhibit high epistemic rationality outside of their immediate areas of expertise is vanishingly small."
I think that there are thousands of people worldwide who exhibit very high epistemic rationality in most domains that they think about. I think that most of these people are invisible owing to the paucity of elites online. I agree that epistemic standards are generally very poor, and that high status academics generally do poorly outside of their immediate areas of expertise.
I should also clarify that I don't think that one needs a silver bullet argument of the type "the people who you would expect to be most trustworthy have the wrong belief on something that they've thought about, with very high probability" to conclude with high confidence that epistemic standards are generally very low.
I think that there are many weak arguments that respected authorities are very often wrong.
Vladimir M has made arguments of the type "there's fierce disagreement among experts at X about matters pertaining to X, so one knows that at least some of them are very wrong." I think that string theory is a good case study. There are very smart people who strongly advocate for string theory as a promising road for theoretical physics research, and other very smart people who strongly believe that the research program is misguided. If nothing else, one can tell that a lot of the actors involved are very overconfident (even if one doesn't know who they are).
In computer science an elite coder might take 6 months to finish a very hard task (e.g. create some kind of tricky OS kernel), but a poor coder will never complete the task. This makes the elite coder infinitely better than the poor coder. Furthermore the poor coder will ask many questions of other people, impacting their productivity. Thus an elite coder is transfinitely more efficient than a poor coder ;-)
That's too strong of a statement. If you exclude some diehards like Deutsch, the Bohmian school and maybe Penrose's gravity-induced decoherence, the prevailing attitude is "however many worlds are out there, we can only ever see one", so, until new evidence comes along, we can safely treat MWI as a single world.
See also Stephen Hsu's comments on this.
Another huge difference is that much of quantum mechanics is very technical physics. To get to the point where you can even have an opinion you need a fair amount of background information. When assessing expert opinion, you have a hugely difficult problem of trying to discern whether an expert physicist has relevant technical knowledge you do not have OR whether they are making a failure in rationality.
This of course is exactly what Muslim theologians would say about Muslim theology. And I'm perfectly happy to say, "Well, the physicists are right and Muslim theologians are wrong", but that's because I'm relying on my own judgment thereon.
The equivalent to asking Muslim theologians about Allah would be to ask many-worlds-believing quantum physicists about many-worlds.
The equivalent of asking quantum physicists about many-worlds would be to ask theologians about Allah, without specifically picking Muslim theologians. And if you ask theologians about Allah (by which I mean the Muslim conception of God--of course "Allah" is just the Arabic for "God"), you're going to find that quite a few of them don't think that Allah exists and that some other version of God does.
And that's not even getting into the problems caused by the fact that religion is a meme that spreads in a way that skews the population of experts, which quantum mechanics doesn't.
Consider a case where you talk to a top flat-earth believer and extract his top 10 reasons for believing the earth is flat.
The act of selecting a Muslim philosopher confounds the accuracy of his belief with the method whereby you selected him. It's like companies searching out a scientist who happens to take a view of some question congenial to that company, then booming his research.
You don't know what's in the QM sequence. The whole point of it (well, one of the whole points) is to show people who wouldn't previously believe such a thing was plausible, that they ought to disagree with elite physicists with high confidence - to break their trust in a sane world, before which nothing can begin.
Does it worry you that people with good domain knowledge of physics(Shminux,Mitchell Porter, myself) seem to feel that your QM sequence is actually presenting a misleading picture of why some elite physicists don't hold to many worlds with high probability?
Also, is it desirable to train rationalists to believe that they SHOULD update their belief about interpretations of quantum mechanics above a weighted sampling of domain experts based on ~50 pages of highschool level physics exposition? I would hope anyone whose sole knowledge of quantum mechanics is the sequence puts HUGE uncertainty bands around any estimate of the proper interpretation of quantum mechanics, because there is so much they don't know (and even more they don't know that they don't know)
Is there an explanation of this somewhere that you can link me to?
This might be relevant.
Not under these circumstances, no. Part of understanding that the world is not sane, is understanding that some people in any given reference class will refuse to be persuaded by any given bit of sanity. It might be worrying if the object-level case against single-world QM were not absolutely clear-cut.
Depends on how crazy the domain experts are being, in this mad world of ours.
Does it worry you, then, that many worlds does not generalize to Lagrangians that lead to non-linear equations of motion, and that many successful particle physics lagrangians are non-linear in the fields?
This sounds wrong. That nonlinearity is in the classical equations of motion. The quantum equations of motion will still be linear, in the sense of obeying the superposition principle. Offhand, I'm not sure how to sum up the quantum consequences of the classical nonlinearity ... maybe something to do with commutators? ... but I don't see this as an argument against MWI.
edit Maybe you mean quadratic fermionic terms? Which I would agree is specifically quantum.
Sure, the equation of motion are non-linear in the FIELDS, which aren't necessarily the wavefunctions. No one has solved Yang-Mills, which is almost certainly the easiest of the non-linear lagrangians, so I don't think we actually know about whether the quantum equations would be linear.
The standard approach is to fracture gauge symmetry and use solutions to the linearized equations of motion as wavefunctions, and treat the non-linear part as an interaction you can ignore at large times. This is actually hugely problematic because Haag's theorem calls into question the entire framework (you can't define an interaction picture).
It seems unlikely that you can have the classical equations of motion be non-linear in the field without the wavefunction having non-linear evolution- after all the creation operator at leading order has to obey the classical equation of motion, and you can write a single particle state as (creation*vacuum). The higher order terms would have to come together rather miraculously.
And keep in mind, its not just Yang-Mills. If we think of the standard model as the power-series of an effective field theory, it seems likely all those linear, first-order equations governing the propagator are just the linearization of the full theory.
There are many arguments against many worlds, I was simply throwing out one that I used to hear bandied about at particle physics conferences that isn't addressed in the sequences at all. And generally, quantum field theories are still on fairly weak mathematical underpinnings. We have a nice collection of recipes that get the job done, but the underlying mathematical structure is unknown. Maybe a miracle occurs. But its a huge unknown area that is worth pointing to as "what can this mean?" Unless someone has dealt with this in the last 5 or 6 years, its been awhile since I was a physicist.
Edit: Replying to your edit, I'm thinking of any higher order terms you can throw into your lagrangian, be they quadratic fermion terms, three or four-point terms in Yang-Mills,etc. These lead to non-linear equations for the fields, and (quite likely, but unproven), the full solution to the wavefunction will be similarly non-linear. This may have implications for whether or not particle-number is a good observable, but I'm tired and don't want to try and work it out.
Edit Again: I re-read this and its pretty clear to me I'm not communicating well (since I'm having trouble understanding what I just wrote). So- I'll try to rephrase. The same non-linearities that crop up in Yang-Mills that require you to pick a proper gauge before you try to use the canonical relationships to write the quantum equations of motion are likely more general. You can add all kinds of non-linear terms to the lagrangian and you can't always gauge fix them away. Most of the time they are small and you can ignore them, but on a fundamental level they are there (and at higher energies then some effective scale they probably matter). This requires modifications to linear quantum mechanics (commutation relationships become power series of which the traditional commutator is just the lowest order term, etc).
Given EHeller's appeal to your authority to support his position in this context this is an interesting development.
It may be worth also observing that at least two of those users have disagreements with you about epistemology and reductionism far more fundamental than QM interpretations. When someone's epistemic philosophy leads them to disagree about existence implications of general relativity then their epistemic disagreement about the implications of QM provides very little additional information.
When you bite the bullet and accept someone else's beliefs based on their authority then consistency suggests you do it at the core point of disagreement, not merely one of the implications thereof. In this case that would require rather sweeping changes.
I'm slightly annoyed that I just reread most of that thread in the understanding that you were linking to the disagreements in question, only to find no comments by either shminux, Mitchell Porter or EHeller and therefore feel no closer to understanding this particular subthread's context.
I think he was referring to shminux's non scientific-realist views, suggesting they are in conflict with such statements as "there are galaxies distant enough that we cannot see them due to lightspeed limitations."
Never mind, that post and thread are far more interesting than the assorted related comments spread over years. Carl's comment is correct and if you want more details about those you may have luck using Wei Dai's script. You'd have to experiment with keywords.
Elite physicists are also people. Would you say that, if exposed to your sequence, these physicists would come to see that they were mistaken in their rejection of MWI? If not, it seems that the most credible explanation of the fact that your sequence can persuade ordinary folk that elite physicists are wrong, but can't persuade elite physicists themselves, is that there is something wrong with your argument, which only elite physicists are competent enough to appreciate.
Certainly many elite physicists were persuaded by similar arguments before I wrote them up. If MWI is wrong, why can't you persuade those elite physicists that it's wrong? Huh? Huh?
Okay, so you are saying that these arguments were available at the time elite physicists made up their minds on which interpretation of QM was correct, and yet only a minority of elite physicists were persuaded. What evidential weight do you assign to this fact? More importantly, what evidential weight should the target audience of your QM sequence assign to it? To conclude that MWI is very likely true after reading your QM sequence, from a prior position of relative agnosticism, seems to me to give excessive weight to my own ability to assess arguments, relative to the ability of people who are smarter than me and have the relevant technical expertise--most of whom, to repeat, were not persuaded by those arguments.
Some kind of community-driven kickstarter to convince a top-level physicist to read the MWI sequence (in return for a grant) and to provide an in-depth answer tailored to it would be awesome. May also be good PR.
Scott Aaronson was already reading along to it as it was published. If we paid David Deutsch to read it, I expect him to just say, "Yeah, that's all basically correct" which wouldn't be very in-depth.
From those who already disagree with MWI, I would expect more in the way of awful amateur epistemology delivered with great confidence. Then those who already had their trust in a sane world broken will nod and say "I expected no better." Others will say, "How can you possibly disregard the word of so great a physicist? Perhaps he knows something you don't!" - though they will not be able to formalize the awful amateur epistemology - and nod among themselves about how Yudkowsky failed to anticipate that so strong a reply might be made (it should be presumed to be a very strong reply since a great physicist made it, even if they can't 100% follow themselves why it is a great refutation, or would not have believed the same words so much from a street performer). And so both will emerge strengthened in their prior beliefs, which isn't much of a test.
Scott Aaronson is not a physicist!
I think the default position isn't that MWI is wrong, but that we don't currently have enough evidence to decide with high confidence. And you could persuade lots of physicists of that.
Unfortunately, in general when someone has given a question lots of thought and come to a conclusion, it will take an absolute steamroller to get them to change their mind. Most elite physicists will have given the question that much thought. So this wouldn't demonstrate as much as you might like.
Elite physicists are easy to persuade in the abstract. You wait till the high status old people die.
-1 for unjustified arrogance. The QM sequence has a number of excellent teaching points, like the discussion of how we can be sure that all electrons are identical, but the idea that one can disagree with experts in a subject matter without first studying the subject matter in depth is probably the most irrational, contagious and damaging idea in all of the sequences.
Example followed. That is, this utterance is poor form and ironic. This charge applies far more to the parent than the grandparent. Advocacy of deference to social status rather than and despite of evidence of competence (or the lack thereof) is the most irrational, contagious and damaging idea that appears on this site. (Followed closely by the similar "but the outside view says <some arbitrary and irrelevant reference class tennis>".)
It seems likely to me that both "the world is insane" and "the world is sane" are incorrect, and the truth is "the world is right about some things and wrong about other things". I like Nick's idea of treating the opinions of people who society regards as experts as a prior and carefully updating from that as evidence warrants. I dislike treating mainstream human civilization as a faction to either align with or break from, and I dislike even more the way some people in the LW community show off by casually disregarding mainstream opinion. This seems like something that both probably looks cultish to outsiders and is legitimately cultish in a way that is bad and worth fighting.
You probably have a good point, but I found it briefly amusing to imagine it going like this:
ELIEZER: Elite scientists are usually elite for good reason, but sometimes they're wrong. People shouldn't blindly trust an elite's position on a subject when they have compelling reasons to believe that that position is wrong.
STRAWMAN: I agree that there are problems with blindly trusting them. But let's not jump straight to the opposite extreme of not blindly trusting them.
As soon as you start saying things like people need "to break their trust in a sane world, before which nothing can begin", you know you have a problem.
I take it you grew up in an atheist or liberal-Christian community?
For example, you may be talking to a child whose parents are still lying to him.
Deciding the world is completely untrustworthy after learning that Santa Claus is a lie seems like the wrong update to make. The right update to make seems to be "adults sometimes lie to kids to entertain themselves and the kids". In fact, arguably society tells you more incorrect info as a kid than as an adult, so you should become more and more trusting of what society tells you the older you grow. (Well, I guess you also get smarter as you grow, so it's a bit more complicated than that.)
You must be strawmanning “the world is insane” if you think it's not compatible with “the world is right about some things and wrong about other things”. EY knows pretty well that the world isn't wrong about everything.
This seems circular :-) (I should use an inside view of your sequence to update the view that I shouldn't give too much weight to my inside view of your sequence...?) But I'll check out your sequence.
That isn't circular. Choosing to limit your use of reasoning to a specific subset of valid reasoning (the assertion of a reference class that you have selected and deference to analogies thereto---'outside view') is a quirk of your own psychology, not a fact about what reasoning is valid or circular. You don't have to believe reasoning that isn't based on the outside view but this is very different from that reasoning being circular. Using arguments that will not convince the particular audience is futile, not fallacious.
The above holds regardless of whether your particular usage of outside view reasoning is sound with respect to the subject discussed in this context. "Circular" is an incorrect description even if Eliezer is wrong.
A related question...
David Hume is the first person who, the way I'm measuring things, was "basically right about nearly all the basics," circa 1740, when his views were mostly minority opinions among elite education opinion. His basic views didn't become majority elite opinion (again, in the way I'm carving things up) until, say, the 1910s. Was Hume justified in being quite confident of his views back in 1740?
I'm not actually asking you to answer: I'm betting neither of us knows enough about Hume or the details of the evidence and argument available to him and his peers to know. But it's an interesting question. I lean non-confidently toward "yes, Hume was justified," but that mostly just reveals my view about elite competence rather than arguing for it.
Information markets have become more efficient over time, and this has asymmetrically improved elite common sense relative to the views of outstanding individuals.
Even if Hume's views were minority opinions, they may not have been flat out rejected by his peers (although they may have been). So the prior against his views being right coming from other people thinking other things may not be that strong.
Even if Hume wouldn't have been justified in believing that the conjunction of all of his views is probably right, he could still have been justified in believing that each individual one is right with high probability.
I think that it's sometimes possible to develop high (~95+%) confidence in views that run counter to elite conventional wisdom. This can sometimes be accomplished by investigating the relevant issues in sufficient detail and by using model combination, as long as one is sufficiently careful about checking that the models that one is using aren't very dependent.
In the particular case of MWI, I doubt that Eliezer has arrived at his view via thorough investigation and by combining many independent models. In a response to Eliezer I highlighted a paper which gives a lot of references to papers criticizing MWI. As I wrote in my comment, I don't think that it's possible to have high confidence in MWI without reading and contemplating these criticisms.
Somehow I had managed to not-think of #1. Thanks.
This reminds me of this passage of Richard Rorty (could not link in a better way, sorry!). Rorty considers whether someone could ever be warranted in believing p against the best, thoughtfully considered opinion of the intellectual elites of his time. He answers:
Now, Rorty is a pragmatist sympathetic to postmodernism and cultural relativism (though without admitting explicitly to the latter), and for him the question is rhetorical and the answer is clearly "no". From the LW point of view, it seems at first glance that the structure of Bayesian probability provides an objective "natural order of reasons" that allows an unequivocal answer. But digging deeper, the problem of the priors brings again the "relativist menace" of the title of Rorty's essay. In practice, our priors come from our biology and our cultural upbringing; even our idealized constructs like Solomonoff induction are the priors that sound most reasonable to us given our best knowledge, that comes largely form our culture. If under considered reflection we decide that the best priors for existing humans involve the elite opinion of existing society, as the post suggests, it follows that Hume (or Copernicus or Einsten!1905 or…) could not be justified in going against that opinion, although they could be correct (and become justified as elites are convinced).
(Note: actually, Einstein might have been justified according to the sophisticated relativism of this post, if he had good reasons to believe that elite opinion would change when hearing his arguments--and he probably did, since the relevant elite opinion did change. But I doubt Hume and Copernicus could have been justified in the same way.)