DSimon comments on David Chalmers' "The Singularity: A Philosophical Analysis" - Less Wrong

33 Post author: lukeprog 29 January 2011 02:52AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (202)

You are viewing a single comment's thread. Show more comments above.

Comment author: DSimon 29 January 2011 11:15:27PM *  3 points [-]

AGI won't automatically converge to Eliezer's "good"

vs.

extrapolated volitions for all evolved agents cohere.

It really really seems like these two statements contradict each other; I think this is the source of the confusion. Can you go into more detail about the second statement?

In particular, why would two agents which both evolved but under two different fitness functions be expected to have the same volition?