Interesting post - while I don't have any real answers I have to disagree with this point:

"Why do you think your computer is not conscious? It probably has more of a conscious experience than, say, a flatworm or sea urchin. (As byrnema notes, conscious does not necessarily imply self-aware here.)"

A computer is no more conscious than a rock rolling down a hill - we program it by putting sticks in the rocks way to guide to a different path. We have managed to make some impressive things using lots of rocks and sticks, but there is not a lot more to it than that in terms of consciousness.

Well, perhaps a bit too simple. Consider this. You set your confidence level at 95% and start throwing a coin. You observe 100 tails out of 100. You publish a report saying "the coin has tails on both sides at a 95% confidence level" because that's what you chose during design. Then 99 other researchers repeat your experiment with the same coin, arriving at the same 95%-confidence conclusion. But you would expect to see about 5 reports claiming otherwise! The paradox is resolved when somebody comes up with a trick using a mirror to observe both sides of the coin at once, finally concluding that the coin is two-tailed with a 100% confidence.

What was the mistake?

The analogy seems pretty nice. The argument seems to be that, based on the historical record, we're doomed to collective inaction in the face of even extraordinarily dangerous risks. I agree that the case of nukes does provide some evidence for this.

I think you paint things a little too grimly, though. We have done at least a little bit to try to mitigate the risks of this particular technology: there are ongoing efforts to prevent proliferation of nuclear weapons and reduce nuclear stockpiles. And maybe a greater risk really would provoke a more serious response.

I've never understood why explaining the Born Rule is less of a problem for any of the other interpretations of QP than it is for MWI. Copenhagen, IIRC, simply asserts it as an axiom. (Rather, it seems to me that MWI is one of the few that even tries to explain it!)

What frightens me is: what if I'm presented with some similar argument, and I can't spot the flaw?

Having recognized this danger, you should probably be more skeptical of verbal arguments.

I see no justification whatsoever for concluding that gravity must therefore be detectable in the weak-field limit.

Suppose we perform an experiment where, based on the measured spin value, we move some macroscopic object with detectable gravity in opposite directions. In the Newtonian background spacetime approach there is no issue with MWI, as both branches live on the same spacetime. In a full GR case, however, the spacetime itself must decohere into different branches, or else we could detect the interaction between different branches gravitationally (I don't know if this has been tested, but it would be extremely surprising if detected). I am not sure what would the mechanism which splits the spacetime itself be, since all current QM/QFT models are done on a fixed background (ignoring ST and LQG). So presumably this requires Quantum Gravity. Yet the whole thing happens at very low energies, slow speeds and weak spacetime curvatures, so that's why I said that this would have to be a QG effect in the weak-field limit. Of course it would only be "detectable" in a sense that if there is no gravitational interaction between branches, then the spacetime itself must decohere by some QG mechanism.

I don't think most of the LW population would regard that as "super high." Plus, most people in the Ivy League having IQs upwards of 130 doesn't equate to most people with IQs upwards of 130 making it into the Ivy League. I'd be interested to know what the correlation with financial success is for additional IQ above the mean among Ivy Leaguers.

Yes, very definitely so. The other thing that makes LW seem... a little bit silly sometimes is the degree of bullet swallowing in the LW canon.

For instance, just today I spent a short while on the internet reading some good old-fashioned "mind porn" in the form of Yves Couder's experiments with hydrodynamics that replicate many aspects of quantum mechanics. This is really developing into quite a nice little subfield, direct physical experiments can be and are done, and it has everything you could want as a reductive explanation of quantum mechanics. Plus, it's actually classical: it yields a full explanation of the real, physical, deterministic phenomena underlying apparently quantum ones.

But if you swallowed your bullet, you'll never discover it yourself. In fact, if you swallow bullets in general, I find it kind of difficult to imagine how you could function as a researcher, given that a large component of research consists of inventing new models to absorb probability mass that currently has nowhere better to go than a known-wrong model.

I don't get it. (I know what random variables and covariance are)

Given that every time we discover something new we find that there are more questions than answers, I find it hard to believe that the process should converge some day.