Cyan comments on Open Thread: November 2009 - Less Wrong

3 [deleted] 02 November 2009 01:18AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (539)

You are viewing a single comment's thread. Show more comments above.

Comment author: Cyan 08 November 2009 01:33:06PM 0 points [-]

How about: an AI can be smart enough to realize all of those things, and it still won't change its utility function. Then link Eliezer's short story about that exact scenario. (Can't find it in two minutes, but it's the one where the dude wakes up with a construct designed to be his perfect mate, and he rejects her because she's not his wife.)

Comment author: Eliezer_Yudkowsky 08 November 2009 07:54:18PM 2 points [-]