You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Nebu comments on Naturalism versus unbounded (or unmaximisable) utility options - Less Wrong Discussion

34 Post author: Stuart_Armstrong 01 February 2013 05:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (72)
Tags: utility infinity paradox resolution determinism ethics choice

Comments (72)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nebu 05 January 2016 08:50:07AM 0 points [-]

Why would we give the AI a utility function that assigns 0 utility to an outcome where we get everything we want but it never turns itself off?

The designer of that AI might have (naively?) thought this was a clever way of solving the friendliness problem. Do the thing I want, and then make sure to never do anything again. Surely that won't lead to the whole universe being tiled with paperclips, etc.

Powered by Reddit Powered by Reddit