hairyfigment comments on Cognitive Neuroscience, Arrow's Impossibility Theorem, and Coherent Extrapolated Volition - Less Wrong

10 Post author: lukeprog 25 September 2011 11:15AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (18)

You are viewing a single comment's thread. Show more comments above.

Comment author: hairyfigment 27 September 2011 07:20:57AM 2 points [-]

I don't think that term means what you think it means. When I try to picture life in a world shaped by CEV, I wind up imagining my new incarnation designing (perhaps as part of a team) visually beautiful ways to harness nearly the full energy of a star.

You seem to assume either that potential knowledge is finite -- which technically seems impossible -- or else that you could never grow into a mind as smart as the FAI. If the second assumption leads to unhappiness the FAI will discover this fact and try to make sure the premise does not hold.