You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Giles comments on The Friendly AI Game - Less Wrong Discussion

38 Post author: bentarm 15 March 2011 04:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (170)

You are viewing a single comment's thread. Show more comments above.

Comment author: Giles 27 April 2011 11:18:28PM 1 point [-]

I had assumed that a new copy of the designer would be spawned for each decision, and shut down afterwards.

Although thinking about it, that might just doom you to a subjective eternity of listening to the AI explain what it's done so far, in the anticipation that it's going to ask you a question at some point.

You'd need a good theory of ems, consciousness and subjective probability to have any idea what you'd subjectively experience.