Manfred comments on Safe questions to ask an Oracle? - Less Wrong

2 Post author: Stuart_Armstrong 27 January 2012 06:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (41)

You are viewing a single comment's thread.

Comment author: Manfred 27 January 2012 08:07:48PM *  2 points [-]

Assuming that you have successfully solved the problem of the AI covering the world with question-askers and that sort of thing:

"How do we write a friendly AI?" would be an important question, if you could trust the answer.

"What could we do that would make it safer for us to operate you?"

"What should we do to start a business that makes lots of money quickly, but doesn't draw peoples' attention to the fact that we have an Oracle AI?" could be useful.

"What are the top risks to the human race in near future? What should we do to reduce them?"

Revolutionize protein synthesis and cure various diseases if you have time to kill.

Comment author: asr 27 January 2012 10:10:52PM 0 points [-]

You ought to ask for not only answers, but evidence. if the goal is provable-friendliness, ask for the proof. In both English and machine-checkable variants.