You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Viliam_Bur comments on In favour of a selective CEV initial dynamic - Less Wrong Discussion

12 [deleted] 21 October 2011 05:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (110)

You are viewing a single comment's thread. Show more comments above.

Comment author: Viliam_Bur 24 October 2011 07:09:19PM *  0 points [-]

How would malevolent aliens behave?

They would kill us, or worse, without giving us any chance.

Just like a super-human AI without design for friendliness will probably kill us, or worse. An AI designed for friendliness will need some choices from us -- for example whether to use CEV of humankind, and how to approximate it if we can't measure literally every person on the planet -- and a bad choice could have horrible consequences.