gRR comments on A taxonomy of Oracle AIs - Less Wrong

13 Post author: lukeprog 08 March 2012 11:14PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (51)

You are viewing a single comment's thread.

Comment author: gRR 09 March 2012 12:59:21PM *  2 points [-]

I think a useful distinction can be made between different "Question-Answerers that cannot answer the 'what ought we to do' question". Although all answers from any such system are unsafe, they can be unsafe in different ways:
(1) unsafe in the sense in which any new human knowledge is unsafe, or
(2) unsafe in specific UFAI-related risk sense, which is much more unsafe.
A system that gives the first kind of answers can be thought of as a domain-specific intelligence enhancer for people.