Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

ChrisHibbert comments on The True Prisoner's Dilemma - Less Wrong

56 Post author: Eliezer_Yudkowsky 03 September 2008 09:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (112)

Sort By: Old

You are viewing a single comment's thread.

Comment author: ChrisHibbert 04 September 2008 12:20:38AM 1 point [-]

In the universe I live in, there are both cooperators and defectors, but cooperators seem to predominate in random encounters. (If you leave yourself open to encounters in which others can choose to interact with you, defectors may find you an easy mark.)

In order to decide how to act with the paperclip maximizer, I have to figure out what kind of universe it is likely to inhabit. It's possible that a random super intelligence from a random universe will have few opportunities to cooperate, but I think it's more likely that there are far more SIs and universes in which cooperation is common.

But even though this is the direct answer to the question EY poses, I think it's more important to point out that his is a better (though not simpler to explain as RH says) depiction of the intended dilemma. It takes much more thought to figure out what about the context would make cooperation reasonable. Viscerally, it's nearly untenable.