Wei_Dai comments on SotW: Check Consequentialism - Less Wrong

38 Post author: Eliezer_Yudkowsky 29 March 2012 01:35AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (311)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nominull 24 March 2012 03:05:19AM 2 points [-]

What, we're not even allowed to have identities now?

Comment author: Wei_Dai 25 March 2012 02:40:29AM 3 points [-]

I previously wrote a comment that seems relevant here:

How to translate identity-based decision making into values and/or beliefs seems non-trivial, and can perhaps be compared to the problem of translating anticipated-reward type decision making into preferences over states of the world or over math.

An agent that lets identity influence its decisions probably deviates from ideal rationality, but how to fix that? If we just excise the identity-based parts of its decision procedure without any compensation, that could easily make it worse off if for example it's CEV depends on its identity.