Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Unknown comments on Something to Protect - Less Wrong

52 Post author: Eliezer_Yudkowsky 30 January 2008 05:52PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (76)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Unknown 31 January 2008 07:12:17PM 0 points [-]

Z. M. Davis, given the existence of that many people, and given that threat, the probability that I personally would be the one threatened in that must be multiplied by one over the number of people, since it could have been anyone else. So the expected disutility from your mugging is one dust speck multiplied by the probability that the Matrix scenario is actually true. This probability is very low, and even if it were unity, the disutility of one dust speck isn't going to get me to pay $100.

So again, I said "without such reasons, it should be accepted." But I have the reasons in this case.

Zubon, of course, Eliezer might well take my second gamble. I was only speaking in preparation for what came ahead: Anon doesn't want to gamble with human lives if there's a small chance of failure. Eliezer may be willing to do this, but once the chance of success becomes extremely small (much smaller than 1 in a billion), as in the Mugging case, he ignores the expected utility, thus falling into exactly the same sort of irrationality as Anon. In relation to this, it is significant that he admitted that he would reject the Mugging even if he had no reason to think that expected utility of rejecting it was greater than the expected utility of accepting it.

Or in other words: expected utility must equal utility times probability, no matter how small the probability is.