The dust speck "dillema" - like a lot of the other exercises that get the mathematically wrong answer from most people is triggering a very valuable heuristic. - The "you are trying to con me into doing evil, so fuck off" Heuristic. Consider the problem as you would of it was a problem you were presented with in real life. The negative utility of the "Torture" choice is nigh-100% certain. It is in your physical presence, you can verify it, and "one person gets tortured" is the kind of event that happens in real life with depressing frequency. The "Billions of people get exposed to very minor annoyance" choice? How is that causal chain supposed to work, anyway? So that choice gets assigned a very high probability of being a lie. And it is the kind of lie people encounter very frequently. False hypotheticals in which large numbers of people suffer if you do not take a certain action are a common lever for cons. From a certain perspective, this is what religion is - Attempts to hack people's utility functions by inserting so absurdly large numbers into the equations so that if you assign any probability at all to them being true they become dominant. So claims that look like this class of attack routinely get assigned a probability of zero unless they have very strong evidence backing them up because that is the only way to defend against this kind of mental malware.

It probably goes like this: "Well, 3^^^3 is a big number; something like 100. Would I torture a person to prevent 100 people having a dust speck in their eyes? How about 200 or 1000? No, this is obviously a madness."