= 27d567bc2d48d9f40e9ccc20ea370b15)
komponisto comments on A Thought on Pascal's Mugging - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (159)
It doesn't quite kill Pascal's mugging - the threat does have to have some minimum level of credibility, but that minimum credibility can still be low enough that hand over the cash. Pascal's mugging only is killed if the expected utility of handing over the cash is negative. To show this I think you really do need to evaluate the probability to the end.
Neither does it kill paperclip maximizers. A bunch of paperclips requires about log2(N) bits to describe, plus the description of the properties of a paperclip. So the paperclip maximizer can still have a constantly-increasing utility as they make more paperclips, your rule would just bound it to growing like log(N).
Good line of thought though: there may still be something in here.
This will be the case in the scenario under discussion, due to the low probability of the mugger's threat (in the "3^^^^3 disutilons" version), or the (relatively!) low disutility (in the "3^^^^3 persons" version, under Michael Vassar's proposal).
Yes; it would be a "less pure" paperclip maximizer, but still an unfriendly AI.
The rule is (proposed to be) necessary for friendliness, not sufficient by any means.