You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

V_V comments on The AI, the best human advisor - Less Wrong Discussion

7 Post author: Stuart_Armstrong 13 July 2015 03:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (15)
Tags: fai reduced_impact wbe uploads ai

Comments (15)

You are viewing a single comment's thread.

Comment author: V_V 09 September 2015 09:31:41AM *  -1 points [-]

So here, the AI merely has to come up with a message that could have come from the WBE, with at least 0.01% probability, and that would indeed make us rich. And, therefore, the message cannot contain secret code that would optimise the universe.

It seems that you are adding unnecessary complexity without actually addressing the core issue.

If you give to the AI the goal "Earn me 10 million dollars/paper clips" then it is unlikely ( * ) that it will do something questionable in order to achieve the goal. If you give it the goal "Maximize the number of dollars/paper clips to my name" then it may start to optimize the universe in ways you won't like. Your trick does nothing to solve this issue.

( * well, provided that you at least tried to implement some moral and legal constraints, otherwise it may well go Walter White)

Comment author: Stuart_Armstrong 09 September 2015 09:39:09AM 0 points [-]

If you give to the AI the goal "Earn me 10 million dollars/paper clips" then it is unlikely that it will do something questionable in order to achieve the goal.

I disagree. In a stochastic universe, you can never be certain you've achieved your goal. An otherwise unrestricted AI with that goal will create a few trillion paper clips. just to be sure, and then obsessively count them again and again. You might make the argument that minor practical restrictions can make that AI design safe, which is plausible, but the goal is not intrinsically safe.

Your trick does nothing to solve this issue.

This is for a reduced impact AI. The idea is to allow a larger impact in one area, without reduced impact shutting it down.

Comment author: V_V 09 September 2015 12:59:04PM -1 points [-]

I disagree. In a stochastic universe, you can never be certain you've achieved your goal. An otherwise unrestricted AI with that goal will create a few trillion paper clips. just to be sure, and then obsessively count them again and again. You might make the argument that minor practical restrictions can make that AI design safe, which is plausible, but the goal is not intrinsically safe.

Under a literal interpretation of the statement it will create exactly 10 million paper clips, not one more. Anyway, nothing is 100% safe.

Comment author: Stuart_Armstrong 09 September 2015 01:08:08PM 0 points [-]

Anyway, nothing is 100% safe.

Yes, but expected utility maximisers of this type will still take over the universe, if they could do it easily, to better accomplish their goals. Reduced impact agents won't.

Powered by Reddit Powered by Reddit