You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Lumifer comments on The Fermi paradox as evidence against the likelyhood of unfriendly AI - Less Wrong Discussion

5 Post author: chaosmage 01 August 2013 06:46PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (53)

You are viewing a single comment's thread. Show more comments above.

Comment author: Lumifer 02 August 2013 03:45:16PM 1 point [-]

It is possible to have factors in one's utility function which limit the expansion.

For example, a utility function might involve "preservation in an untouched state", something similar to what humans do when they declare a chunk of nature to be a protected wilderness.

Or a utility function might contain "observe development and change without influencing it".

And, of course, if we're willing to assume an immutable cast-in-stone utility function, why not assume that there are some immutable constraints which go with it?