You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Grognor comments on Safe questions to ask an Oracle? - Less Wrong Discussion

2 Post author: Stuart_Armstrong 27 January 2012 06:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (41)

You are viewing a single comment's thread.

Comment author: Grognor 27 January 2012 07:09:16PM 0 points [-]

The obvious and almost certainly correct answer to the meta-question is that there is no question that can be asked that won't cause the AI to devote as many possible resources to answering forever, given that activating it doesn't cause it to ask itself the simplest possible question and then spend eternity answering it over and over without ever interacting non-harmfully with humans.