Kaj_Sotala comments on An attempt to dissolve subjective expectation and personal identity - LessWrong

35 Post author: Kaj_Sotala 22 February 2013 08:44PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (68)

You are viewing a single comment's thread. Show more comments above.

Comment author: Kaj_Sotala 28 February 2013 09:23:23AM *  0 points [-]

My claim is that it doesn't seem complete to argue "we need a sense of identity to run long-run optimization problems well." I run optimization programs without a sense of identity just fine- you tell them the objective function, you tell them the decision variables, you tell them the constraints, and then they process until they've got an answer. It doesn't seem to me like you're claiming the 'sense of personal identity' boils down to 'the set of decision variables and the objective function,' but I think that's only as far as your argument goes.

Hmm, looks like I expressed myself badly, as several people seem to have this confusion. I wasn't saying that long-term optimization problems in general would require a sense of identity, just that the specific optimization program that's implemented in our current mental architecture seems to require it.

(Yes, a utilitarian could in principle decide that they want to minimize the amount of suffering in the world and then do a calculation about how to best achieve that which didn't refer to a sense of identity at all... but they'll have a hard time getting themselves to actually take action based on that calculation, unless they can somehow also motivate their more emotional predictive systems - which are based on a sense of personal identity - to also be interested in pursuing those goals.)