TRIZ-Ingenieur comments on Superintelligence 11: The treacherous turn - Less Wrong

10 Post author: KatjaGrace 25 November 2014 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (50)

You are viewing a single comment's thread. Show more comments above.

Comment author: TRIZ-Ingenieur 25 November 2014 10:56:05PM 2 points [-]

What about hard wired fears, taboos and bad conscience triggers? Recapitulating Omohundro "AIs can monitor AIs" - assume to implement conscience as an agent - listening to all thoughts and taking action in case. For safety reasons we should educate this concience agent with utmost care. Conscience agent development is an AI complete problem. After development the conscience functionality must be locked against any kind of modification or disabling.

Comment author: Liso 26 November 2014 10:28:24AM 0 points [-]

Positive emotions are useful too. :)