Wei_Dai comments on Outlawing Anthropics: An Updateless Dilemma - Less Wrong

26 Post author: Eliezer_Yudkowsky 08 September 2009 06:31PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (194)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 10 September 2009 07:28:25AM 3 points [-]

if two copies of you play a game, the correct behavior for both of them would be to try to win, regardless of what overall outcome you'd prefer before the copying

That doesn't make sense to me, unless you're assuming that the player isn't capable of self-modification. If it was, wouldn't it modify itself so that its copies won't try to win individually, but cooperate to obtain the outcome that it prefers before the copying?

Comment author: cousin_it 10 September 2009 08:24:15AM *  2 points [-]

Yes, that's right. I've shifted focus from correct program behavior to correct human behavior, because that's what everyone else here seems to be talking about. If the problem is about programs, there's no room for all this confusion in the first place. Just specify the inputs, outputs and goal function, then work out the optimal algorithm.