You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

roystgnr comments on The germ of an idea - Less Wrong Discussion

6 Post author: Stuart_Armstrong 13 November 2014 06:58PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (11)

You are viewing a single comment's thread.

Comment author: roystgnr 13 November 2014 10:07:17PM 1 point [-]

the same idea can make an Oracle not attempt to manipulate us through its answers, by making it indifferent as to whether the message was read.

This sounds like an idea from Wei Dai 2 years ago. The specific concern I had then was that the larger the Oracle's benefit to humanity, the less relevant predictions of the form "what would happen if nobody could read the Oracle's prediction" become.

The general concern is the same as with other Oracle AIs: a combination of an Oracle and the wrong human(s) looks an awful lot like an unfriendly AI.