PhilGoetz comments on The scourge of perverse-mindedness - Less Wrong

95 Post author: simplicio 21 March 2010 07:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (249)

You are viewing a single comment's thread. Show more comments above.

Comment author: PhilGoetz 21 March 2010 11:37:27PM *  4 points [-]

It is probably possible to pass that test by exploiting human psychology. It is probably impossible to do well on that test by trying to convince humans that your viewpoint is right.

You're talking past orthonormal. You're assuming a properly-designed AI. He's saying that accomplishing the task would be strong evidence of unfriendliness.