You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

hairyfigment comments on Cognitive Neuroscience, Arrow's Impossibility Theorem, and Coherent Extrapolated Volition - Less Wrong Discussion

10 Post author: lukeprog 25 September 2011 11:15AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (18)

You are viewing a single comment's thread. Show more comments above.

Comment author: hairyfigment 27 September 2011 07:20:57AM 2 points [-]

I don't think that term means what you think it means. When I try to picture life in a world shaped by CEV, I wind up imagining my new incarnation designing (perhaps as part of a team) visually beautiful ways to harness nearly the full energy of a star.

You seem to assume either that potential knowledge is finite -- which technically seems impossible -- or else that you could never grow into a mind as smart as the FAI. If the second assumption leads to unhappiness the FAI will discover this fact and try to make sure the premise does not hold.