Pavitra comments on Anyone read Erfworld? - Less Wrong

22 Post author: Raemon 01 April 2011 01:48AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (30)

You are viewing a single comment's thread. Show more comments above.

Comment author: Pavitra 02 April 2011 12:07:41AM *  9 points [-]

It doesn't matter if he could escape if he wanted to, because he couldn't want to escape unless he already had done so. Friendliness is stable under self-modification.

Edit: I managed to type that whole thing without quite realizing how perfect Parson is as a lay-accessible model for what an alien intelligence looks like. The perversity of his ingenuity, stemming from the fact that he doesn't share the prejudices of the people around him, is a major part of what people fail to anticipate in AI.