= 27d567bc2d48d9f40e9ccc20ea370b15)
JoshuaZ comments on To signal effectively, use a non-human, non-stoppable enforcer - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (164)
Wow! Now that you mention that article, I think I had solved the unsolved problem Eliezer describes in it, back in a discussion from a month ago, not realizing that my position on it wasn't the standard one here!
Someone tell me if I'm missing something here: Eliezer is saying that utility that a hypothesis predicts (from a course of action) can increase much faster than the length of the hypothesis. Therefore, you could feed an ideal AI a prediction that is improbable, but with a large enough utility to make it nevertheless highly important. This would force the AI to give in to Pascal's muggings.
My response (which I assumed was the consensus!) was that, when you permit a hypothesis long enough to associate that mega-utility with that course of action, you are already looking at very long hypotheses. When you allow all of those into consideration, you will necessarily allow hypotheses with similar probability but believe the opposite utility from that COA.
Because the mugger has not offered evidence to favor his/her hypothesis over the opposite, you assign, on net, no significant expected (dis)utility to what the mugger claims to do.
But that's easy to solve. If you've already seen evidence that the mugger is someone who strongly keeps promises then you've now have enough reason to believe them to put the direction in favor of the mugger releasing the the AI. One doesn't necessarily even need that because humans more often tell the truth than lie, and more often keep their promises than break them. Once the probability of the mugger doing what they threaten is a tiny bit over 1/2, Pascal's mugging still is a threat.
Maybe not. Game theoretically, making yourself visibly vulnerable to Pascal's Muggings may guarantee that they will occur, making them cease to constitute evidence.
I've actually just expanded on this idea in the original Pascal's Mugging article. If the Mugger's claims are in no way associated with you or similar muggings, then conceivably you should take the probability at face value. But if that's not the case, then the probability of a direct manipulation attempt should also be taken into consideration, negating the increase in claimed utility.
I think that solves it.