You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Stuart_Armstrong comments on The AI, the best human advisor - Less Wrong Discussion

7 Post author: Stuart_Armstrong 13 July 2015 03:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (15)
Tags: fai reduced_impact wbe uploads ai

Comments (15)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 09 September 2015 09:39:09AM 0 points [-]

If you give to the AI the goal "Earn me 10 million dollars/paper clips" then it is unlikely that it will do something questionable in order to achieve the goal.

I disagree. In a stochastic universe, you can never be certain you've achieved your goal. An otherwise unrestricted AI with that goal will create a few trillion paper clips. just to be sure, and then obsessively count them again and again. You might make the argument that minor practical restrictions can make that AI design safe, which is plausible, but the goal is not intrinsically safe.

Your trick does nothing to solve this issue.

This is for a reduced impact AI. The idea is to allow a larger impact in one area, without reduced impact shutting it down.

Comment author: V_V 09 September 2015 12:59:04PM -1 points [-]

I disagree. In a stochastic universe, you can never be certain you've achieved your goal. An otherwise unrestricted AI with that goal will create a few trillion paper clips. just to be sure, and then obsessively count them again and again. You might make the argument that minor practical restrictions can make that AI design safe, which is plausible, but the goal is not intrinsically safe.

Under a literal interpretation of the statement it will create exactly 10 million paper clips, not one more. Anyway, nothing is 100% safe.

Comment author: Stuart_Armstrong 09 September 2015 01:08:08PM 0 points [-]

Anyway, nothing is 100% safe.

Yes, but expected utility maximisers of this type will still take over the universe, if they could do it easily, to better accomplish their goals. Reduced impact agents won't.

Powered by Reddit Powered by Reddit