Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Strange7 comments on Domesticating reduced impact AIs - Less Wrong

9 Post author: Stuart_Armstrong 14 February 2013 04:59PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (104)

You are viewing a single comment's thread. Show more comments above.

Comment author: Strange7 21 February 2013 03:54:04PM 0 points [-]

It's a minus if you're trying to convince someone more results-oriented to keep giving you R&D funding. Imagine the budget meeting:

The EPA is breathing down our necks about venting a billion dollars worth of antimatter, you've learned literally nothing, and you consider that a good outcome?

If the AI is indifferent to future outcomes, what stops it from manipulating those outcomes in whatever way is convenient for it's other goals?

Comment author: Stuart_Armstrong 21 February 2013 04:44:38PM 0 points [-]

Indifference means that it cannot value any change to that particular outcome. More details at: http://www.fhi.ox.ac.uk/__data/assets/pdf_file/0020/18371/2010-1.pdf