Today's post, Pascal's Mugging: Tiny Probabilities of Vast Utilities was originally published on 19 October 2007. A summary (taken from the LW wiki):
An Artificial Intelligence coded using Solmonoff Induction would be vulnerable to Pascal's Mugging. How should we, or an AI, handle situations in which it is very unlikely that a proposition is true, but if the proposition is true, it has more moral weight than anything else we can imagine?
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was "Can't Say No" Spending, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
It's not at all the same. This is not a problem invoking Omega. If you want that go to the lifespan dilemma.
If we know Omega has 3^^^^3 sided dice and will kill the people if it lands on the one, then I'd shut up and calculate.
Pascal's wager involves much more uncertainty than that. It involves uncertainty about the character speaking. Once a being is claiming it has magic and wants you to do something, to the extent one believes the magic part, one loses one's base of reference to judge the being as truthful, non-whimsical, etc.
Are you arguing that he's more likely to torture them if you give him the money, that the probabilities are the same to within one part in 3^^^^3, or that since it's not a dice, probability works fundamentally differently?
My response was assuming the first. The second one is ridiculous, and I don't think anyone would consider that if it weren't for the bias of giving round numbers for probabilities. If it's the third one, I'd suggest reading probability is in the mind. You don't know which side the die will land on, this is no different than not knowing what kind of a person the character is.