Stuart_Armstrong comments on Domesticating reduced impact AIs - All

9 Post author: Stuart_Armstrong 14 February 2013 04:59PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (104)
Tags: algorithm prediction safety reduced_impact risk mathematics ai

Comments (104)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 18 February 2013 12:58:52PM 1 point [-]

self-destruct in some low-collateral-damage way which prevents anything interesting from being learned by analysis of the remains

That's a plus, not a minus.

We can also use utility indifference (or something analogous) to get some useful info out.

Comment author: Strange7 21 February 2013 03:54:04PM 0 points [-]

It's a minus if you're trying to convince someone more results-oriented to keep giving you R&D funding. Imagine the budget meeting:

The EPA is breathing down our necks about venting a billion dollars worth of antimatter, you've learned literally nothing, and you consider that a good outcome?

If the AI is indifferent to future outcomes, what stops it from manipulating those outcomes in whatever way is convenient for it's other goals?

Comment author: Stuart_Armstrong 21 February 2013 04:44:38PM 0 points [-]

Indifference means that it cannot value any change to that particular outcome. More details at: http://www.fhi.ox.ac.uk/__data/assets/pdf_file/0020/18371/2010-1.pdf

Powered by Reddit Powered by Reddit