= 27d567bc2d48d9f40e9ccc20ea370b15)
Sewing-Machine comments on A Thought on Pascal's Mugging - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (159)
Given that there's no definition for the value of a util, arguments about how many utils the universe contains aren't likely to get anywhere.
So let's make it easier. Suppose the mugger asks you for $1, or ey'll destroy the Universe. Suppose we assume the Universe to have 50 quadrillion sapient beings in it, and to last for another 25 billion years ( = 1 billion generations if average aliens have similar generation time to us) if not destroyed. That means the mugger can destroy 50 septillion beings. If we assign an average being's life as worth $100000, then the mugger can destroy $5 nonillion (= 5 * 10^30).
Given that there have been reasonable worries about ie the LHC destroying the Universe, I think the probability that a person can destroy the universe is rather greater than 1 in 5 nonillion (to explain why it hasn't been done already, assume the Great Filter comes at the stage of industrialization). I admit that the probability of someone with an LHC-level device being willing to destroy the Universe for the sake of $1 would be vanishingly low, but until today I wouldn't have thought someone would kill 6,790 people to protest a blog's comment policy either.
But Komponisto's idea is not to do with how many utils the universe contains.
Incidentally, it seems to me that if it's possible to make a credible threat to destroy the universe, then our main problem is not Pascal's mugging but the fragility of the universe.
As I understand it, komponisto's idea is that we don't have to worry about Pascal's Mugging because the probability of anyone being able to control 3^^^^3 utils is even lower than one would expect simply looking at the number 3^^^^3, and is therefore low enough to cancel out even this large a number.
What I am trying to respond is that there are formulations of Pascal's Mugging which do not depend on the number 3^^^^3. The idea that someone could destroy a universe worth of utils is more plausible than destroying 3^^^^3 utils, and it's not at all obvious there that the low probability cancels out the high risk.
Well, it may not be obvious what to do in that case! But the original formulation of the Pascal's Mugging problem, as I understand it, was to formally explain why it is obvious in the case of large numbers like 3^^^^3:
The answer proposed here is that a "friendly" utility function does not in fact allow utility to increase faster than complexity increases.
I don't claim this tells us what to do about the LHC.