komponisto comments on A Thought on Pascal's Mugging - Less Wrong

12 Post author: komponisto 10 December 2010 06:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (159)

You are viewing a single comment's thread. Show more comments above.

Comment author: komponisto 11 December 2010 06:57:45AM 0 points [-]

Pascal's mugging only is killed if the expected utility of handing over the cash is negative.

This will be the case in the scenario under discussion, due to the low probability of the mugger's threat (in the "3^^^^3 disutilons" version), or the (relatively!) low disutility (in the "3^^^^3 persons" version, under Michael Vassar's proposal).

So the paperclip maximizer can still have a constantly-increasing utility as they make more paperclips, your rule would just bound it to growing like log(N)

Yes; it would be a "less pure" paperclip maximizer, but still an unfriendly AI.

The rule is (proposed to be) necessary for friendliness, not sufficient by any means.