= 27d567bc2d48d9f40e9ccc20ea370b15)
Yvain comments on A Thought on Pascal's Mugging - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (159)
Given that there's no definition for the value of a util, arguments about how many utils the universe contains aren't likely to get anywhere.
So let's make it easier. Suppose the mugger asks you for $1, or ey'll destroy the Universe. Suppose we assume the Universe to have 50 quadrillion sapient beings in it, and to last for another 25 billion years ( = 1 billion generations if average aliens have similar generation time to us) if not destroyed. That means the mugger can destroy 50 septillion beings. If we assign an average being's life as worth $100000, then the mugger can destroy $5 nonillion (= 5 * 10^30).
Given that there have been reasonable worries about ie the LHC destroying the Universe, I think the probability that a person can destroy the universe is rather greater than 1 in 5 nonillion (to explain why it hasn't been done already, assume the Great Filter comes at the stage of industrialization). I admit that the probability of someone with an LHC-level device being willing to destroy the Universe for the sake of $1 would be vanishingly low, but until today I wouldn't have thought someone would kill 6,790 people to protest a blog's comment policy either.
Citation needed.
Looking at:
http://en.wikipedia.org/wiki/Safety_of_particle_collisions_at_the_Large_Hadron_Collider
...the defenders are doing a PR whitewash job. They can't even bring themselves to mention probabilities!
Maybe its because there would be no point to mentioning probabilities smaller than e^(-10^9) (the evidence you get from the fact that the sun still exists) citation, since humans don't deal well with small numbers.
But no, "whitewash job" :P
IMO, this is most likely to do with the percieved difference between "no risk" and "some risk". I am sure the authors were capable of producing a quantitative report - and understand that that is the scientific approach - but sat on any figures they might have had - after being instructed about the presentation of the desired conclusion.
This sounds a bit conspiracy-ey. Any evidence for your claims, e.g. a trend of similar papers using probability assessments rather than just stopping at "these collisions have happened a very large number of times and we ain't dead yet"?
Risk assessments are commonly quantitave.
Fair enough. So we might have enough data for the analysis. But "are commonly quantitative" isn't even weak evidence either way - that is to say, this paper being less quantitative doesn't ring any alarm bells per se, since it's not unusual. But we can get evidence by looking closer: are qualitative risk assessments more likely to be "instructed about the desired conclusion" than quantitative ones? What complicating variables can we prune out to try and get the causal relationship whitewash->qualitative?
Basically what I'm trying to communicate is that there are two ways you could convince me this was a fraud: you could have better knowledge of the subject matter than me and demonstrate directly how it was a fraud, or you could have detailed evidence on frauds, good enough to overcome my prior probability that this isn't a fraud. Saying "they were probably able to produce a more quantitative report, but didn't, so it's a fraud" is neither.
I never used the term "fraud". You seem to be reading more into this than was intended. I just think it is funny that an official LHC risk assessment paper presumably designed to reassure fails to come up with any probabilities - and just says: "it's safe". To someone like me, that makes it look as though it is primarily a PR exercise.
IIRC, others have observed this before me - though I don't have the reference handy.
I would classify a supposedly scientific paper that "sat on figures" and "was instructed about the desired conclusion" as a fraud. If you would prefer "whitewash" (a word you did use) instead of "fraud" I would be happy to change in the future.
But the paper was quite a bit longer than "it's safe," seemed quite correct (though particle physics isn't my field), and indeed gave you enough information to calculate approximate probabilities yourself if you wanted to. So to me it looks like you're judging on only a tiny part of the information you actually have.
Because it doesn't actually say the words "not greater than 1 in 3E22 and that's just calculating using the cosmic rays that have hit the earth in the last 4.5E9 years" means it should be ignored?
I am most disappointed the Brian Cox quote didn't make it into that article. The quote was actually newsworthy, too.
Lifeboat Foundation fits my criteria of "reasonable", as do some of the commenters here. Even if there's only a one in a million risk of destroying the world, that's still equivalent to killing 6,000 people with probability one; potentially destroying the Universe should require even more caution.
There's not even a one in a million; it's closer to "But there's still a chance, right?"
And you're still dealing in probabilities too small to sensibly calculate in this manner and be saying anything meaningful - "switching on the LHC is equivalent to killing 6,000 people for certain" is a statement that isn't actually sensible when rendered in English, and I don't see another way to render in English your calculated result that switching it on is "equivalent to killing 6,000 people with probability one". But please do enlighten me.
(I realise you're multiplying 6E9 by 1E-6 and asserting that six billion conceptual millionth-of-a-person slivers equals six thousand actual existing people. "Shut up and multiply" doesn't stop me balking at this, and that the result says "switching on the LHC is equivalent to killing 6,000 people for certain" seems to constitute a reductio ad absurdum for however one gets there.)
Rees estimated the probability of the LHC destroying the world at 1 in 50 million, and it would be surprising if he were one of the few people in the world without overconfidence bias, or one of the few people in the world who doesn't underestimate global existential risks.
I assume from the first sentence that you believe an appropriate probability to have for the LHC destroying the world is less than one in a billion. Trusting anyone, even the world scientific consensus, with one in a billion probability, seems excessive to me - the world scientific consensus has been wrong on more than one in every billion issues it thinks it's sure about. If you're working not off the world scientific consensus but off your own intuition, that seems even stranger - if, for example, the LHC will destroy the world if and only if strangelets are stable at 10 TeEV, then you just discovered important properties about the stability of strangelets to p = < .000000001 certainty, which seems like the sort of thing you shouldn't be able to do without any experiments or mathematics. If you're working off of a general tendency for the world not to be destroyed, well, there were five mass extinction events in the past billion years, so ignoring for the moment the tendency of mass extinctions to take multiple years, that means the probability of a mass extinction beginning in any particular year is about 5/billion. If I were to tell you "The human race will become extinct the year the LHC is switched on", would you really tell me "Greater than 80% chance it has nothing to do with the LHC" and go about your business?
I am still uncomfortable with the whole "shut up and multiply" concept too. But I think that's where the "shut up" part comes in. You don't have to be comfortable with it. You don't have to like it. But if the math checks out, you just shut up and keep your discomfort to yourself, because math is math and bad things happen when you ignore it.
Here we run into the problem of "garbage in, garbage out."
He assigned 50% extinction risk for the 21st century in his book. His overall estimates of risk are quite high.
What your probability discussion there seems to me to be saying is "these numbers are too small to think about in any sensible way, let alone calculate." Trying to think about them closely resembles an argument that the way to deal with technological existential risk is to give up technology and go back to the savannah (caves are too techy).
But the math leads to statements like "switching on the LHC is equivalent to killing 6,000 people for certain", which seems to constitute a reductio ad absurdum of whatever process led to such a sentence.
(You could justify it philosophically, but you're likely to get an engineer's answer: "No it isn't. Here, I'll show you. (click) Now, how many individuals did that just kill?")
One day I would like to open up an inverse casino.
The inverse casino would be full of inverse slot machines. Playing the inverse slot machines costs negative twenty-five cents - that is, each time you pull the bar on the machine, it gives you a free quarter. But once every few thousand bar pulls, you will hit the inverse jackpot, and be required to give the casino several thousand dollars (you will, of course, have signed a contract to comply with this requirement before being allowed to play).
You can also play the inverse lottery. There are ten million inverse lottery tickets, and anyone who takes one will get one dollar. But if your ticket is drawn, you must pay me fifteen million dollars. If you don't have fifteen million dollars, you will have various horrible punishments happen to you until fifteen million dollars worth of disutility have been extracted from you.
If you believe what you are saying, it seems to me that you should be happy to play the inverse lottery, and believe there is literally no downside. And it seems to me that if you refused, I could give you the engineer's answer "Look, (buys ticket) - a free dollar, and nothing bad happened to me!"
And if you are willing to play the inverse lottery, then you should be willing to play the regular lottery, unless you believe the laws of probability work differently when applied to different numbers.
The hedge fund industry called. They want their idea of selling far out-of-the-money options back.
Doesn't this describe the standard response to cars?
Just think of all the low-probability risks cars subsume! Similarly, if you take up smoking you no longer need to worry about radon in your walls, pesticides in your food, air pollution or volcano dust. It's like a consolidation loan! Only dumber.
Sorry, I don't understand. Response to cars?
Most of life is structured as a negative lottery. You get in a car, you get where you're going much faster- but if the roulette ball lands on 00, you're in the hospital or dead. (If it only lands on 0, then you're just facing lost time and property.)
And so some people are mildly afraid of cars, but mostly people are just afraid of bad driving or not being in control- the negative lottery aspect of cars is just a fact of life, taken for granted and generally ignored when you turn the key.
This is plausible and I shall contemplate it.
By the way, and a little bit on topic, I think it's not a coincidence that an inverse casino would be more expensive to run than a regular casino.
What Sewing-Machine said. A solution of the Pascal's mugging problem certainly doesn't imply that existential risks aren't to be worried about!
But Komponisto's idea is not to do with how many utils the universe contains.
Incidentally, it seems to me that if it's possible to make a credible threat to destroy the universe, then our main problem is not Pascal's mugging but the fragility of the universe.
As I understand it, komponisto's idea is that we don't have to worry about Pascal's Mugging because the probability of anyone being able to control 3^^^^3 utils is even lower than one would expect simply looking at the number 3^^^^3, and is therefore low enough to cancel out even this large a number.
What I am trying to respond is that there are formulations of Pascal's Mugging which do not depend on the number 3^^^^3. The idea that someone could destroy a universe worth of utils is more plausible than destroying 3^^^^3 utils, and it's not at all obvious there that the low probability cancels out the high risk.
Well, it may not be obvious what to do in that case! But the original formulation of the Pascal's Mugging problem, as I understand it, was to formally explain why it is obvious in the case of large numbers like 3^^^^3:
The answer proposed here is that a "friendly" utility function does not in fact allow utility to increase faster than complexity increases.
I don't claim this tells us what to do about the LHC.