You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Stuart_Armstrong comments on CEV-inspired models - Less Wrong Discussion

7 Post author: Stuart_Armstrong 07 December 2011 06:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (41)
Tags: cev ai decisions coherent extrapolated utility

Comments (41)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 08 December 2011 09:41:40AM 1 point [-]

For example, suppose my utility function is U(Universe) = #paperclips, which is unbounded in a big universe. Then you're going to normalise me as assigning U(AI becomes clippy) = 1, and U(individual paperclips) = 0.

Yep.

So most likely a certain proportion of the universe will become paperclips.

Powered by Reddit Powered by Reddit