Followup to: When (Not) To Use Probabilities, How Many LHC Failures Is Too Many?
While trying to answer my own question on "How Many LHC Failures Is Too Many?" I realized that I'm horrendously inconsistent with respect to my stated beliefs about disaster risks from the Large Hadron Collider.
First, I thought that stating a "one-in-a-million" probability for the Large Hadron Collider destroying the world was too high, in the sense that I would much rather run the Large Hadron Collider than press a button with a known 1/1,000,000 probability of destroying the world.
But if you asked me whether I could make one million statements of authority equal to "The Large Hadron Collider will not destroy the world", and be wrong, on average, around once, then I would have to say no.
Unknown pointed out that this turns me into a money pump. Given a portfolio of a million existential risks to which I had assigned a "less than one in a million probability", I would rather press the button on the fixed-probability device than run a random risk from this portfolio; but would rather take any particular risk in this portfolio than press the button.
Then, I considered the question of how many mysterious failures at the LHC it would take to make me question whether it might destroy the world/universe somehow, and what this revealed about my prior probability.
If the failure probability had a known 50% probability of occurring from natural causes, like a quantum coin or some such... then I suspect that if I actually saw that coin come up heads 20 times in a row, I would feel a strong impulse to bet on it coming up heads the next time around. (And that's taking into account my uncertainty about whether the anthropic principle really works that way.)
Even having noticed this triple inconsistency, I'm not sure in which direction to resolve it!
(But I still maintain my resolve that the LHC is not worth expending political capital, financial capital, or our time to shut down; compared with using the same capital to worry about superhuman intelligence or nanotechnology.)
"Then, I considered the question of how many mysterious failures at the LHC it would take to make me question whether it might destroy the world/universe somehow, and what this revealed about my prior probability..."
From the previous thread:
"Inevitably, many commenters said, "Anthropic principle! If the LHC had worked, it would have produced a black hole or strangelet or vacuum failure, and we wouldn't be here!"... After observing empirically that the LHC had failed 100 times in a row, would you endorse a policy of keeping the LHC powered up, but trying to fire it again only in the event of, say, nuclear terrorism or a global economic crash?"
If the LHC fails repeatedly it can only be because of logical engineering flaws. In fact, the complexity of the engineering makes it easier for people to attribute failures to unseen, unnatural forces.
If a marble rolling down an incline could destroy the universe, the unnaturalness of the failures could not be hidden. Any incline you approached with marble-y intent would crumble to dust. Or instead of rolling down an incline, the marble would hover in midair.
If the LHC is a machine based on know physics and mechanics then it would require causality-defying forces to stop it from working, just as it would take supernatural forces to stop anyone from simply rolling a marble down a slope.
And if this is the case, why should the LHC supernatural stop-gaps appear as they do -- as comprehensible engineering flaws? Why not something more unambiguously causality-defying like the LHC floating into the air and then disappearing into the void in an exciting flash of lights? Or why not something more efficient? Why should the machine even be built up to this point, only to be blocked by thousands of suspiciously impish last-minute flaws, when a million reasonable legislative, cooperative, or cognitive events could have snuffed the machine from ever even being considered in the first place?
More importantly, if the reasoning here is that some epic force puts the automatic smack-down on any kind of universe destroying event, then, obviously, repeated probability-defying failures of the LHC more logically reduces the probability that it will destroy the universe (by lending increasing support to the existence of this benign universe-preserving force). It doesn't increase the probability of destruction, by its own internal logic.
The argument for stopping the LHC then could only be economic, not self-preservational. So nuclear terrorism would actually be the worse time to start it up, since the energy and man-power resources would be needed more for pressing survival goals, then to operate a worthless machine the universe won't allow us to use.
Very belated note: the human brain is a much more volatile system than millions of tons of material. I suspect that if the LHC could have ended up destroying the universe, the idea to build one would never have occurred to us.