Open Thread, November 23-30, 2013

passive_fist

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

The dust speck "dillema" - like a lot of the other exercises that get the mathematically wrong answer from most people is triggering a very valuable heuristic. - The "you are trying to con me into doing evil, so fuck off" Heuristic.
Consider the problem as you would of it was a problem you were presented with in real life.

The negative utility of the "Torture" choice is nigh-100% certain. It is in your physical presence, you can verify it, and "one person gets tortured" is the kind of event that happens in real life with depressing frequency. The "Billions of people get exposed to very minor annoyance" choice? How is that causal chain supposed to work, anyway? So that choice gets assigned a very high probability of being a lie.

And it is the kind of lie people encounter very frequently. False hypotheticals in which large numbers of people suffer if you do not take a certain action are a common lever for cons. From a certain perspective, this is what religion is - Attempts to hack people's utility functions by inserting so absurdly large numbers into the equations so that if you assign any probability at all to them being true they become dominant.

So claims that look like this class of attack routinely get assigned a probability of zero unless they have very strong evidence backing them up because that is the only way to defend against this kind of mental malware.

This is essentially an outside-an-argument argument. If we really had a choice between 50 years of torture and 3^^^3 dust specks, the rational choice would be the 50 years of torture. But the probability of this description of the situation being true, is extremely low.

If you, as a human, in a real-life situation believe that you are choosing between 50 years of torture and 3^^^3 dust specks, almost certainly you are confused or insane. There will not be the 3^^^3 dust specks, regardless of whichever clever argument has convinced you so; you are choosing b... (read more)

2A1987dM12y

When the prior probability is exactly zero, the posterior probability will still be exactly zero no matter how strong the evidence is.

0DanielLC12y

If you were presented it in real life, there would be a lot fewer than 3^^^3 people at stake. You don't need to consider certainty. 50 years of torture is also something you won't encounter in real life, but you can get a lot closer.