Nebu comments on Naturalism versus unbounded (or unmaximisable) utility options - Less Wrong

34 Post author: Stuart_Armstrong 01 February 2013 05:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (72)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nebu 05 January 2016 08:50:07AM 0 points [-]

Why would we give the AI a utility function that assigns 0 utility to an outcome where we get everything we want but it never turns itself off?

The designer of that AI might have (naively?) thought this was a clever way of solving the friendliness problem. Do the thing I want, and then make sure to never do anything again. Surely that won't lead to the whole universe being tiled with paperclips, etc.