AdeleneDawner comments on Meditation, insight, and rationality. (Part 1 of 3) - Less Wrong

35 Post author: DavidM 28 April 2011 08:26PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (120)

You are viewing a single comment's thread. Show more comments above.

Comment author: MinibearRex 29 April 2011 08:17:13PM 2 points [-]

I did not communicate what I meant to say very well. I'll try again.

I view my utility function as a mathematical representation of all of my own preferences. My working definition of "preferences" is: conditions that, if satisfied by the universe, will cause me to feel good about the universe's state, and if unsatisfied by the universe, will cause me to feel bad about the universe's state. When I talk about "feeling good" and "feeling bad" in this context, I'm trying to refer to whatever motivation it is that causes us to try to maximize what we call "utility". I don't know a good way in english to differentiate between the emotion that is displayed, for instance when a person is self flagellating, and the emotion that causes someone to try to take down a corrupt ruler.

If I learn that some dictator ruling over some other country is torturing and killing that country's people, my internal stream of consciousness may register the statement, "That is not acceptable. What should I do to try to improve the situation of the people in that country?" That is a negative "feeling" produced by the set of preferences that I label "morality". I do not particularly want the parts of my brain that make me moral to vanish. I do not want to self modify in such a way that I will genuinely have no preference between a world where the leader of country X is committing war crimes, and a world where country X is at peace and the population is generally well off.

Should I mope around and feel terrible because the citizens of country X are being tortured? Of course not. That's unhelpful. I do not, in fact, have a positive term in my utility function for my own misery, as my earlier post, now that I've reread it, seems to imply. Rather, I have a positive term in my utility function for whatever it is that doesn't make a person a sociopath, and that was what I was trying to talk about.

Comment author: AdeleneDawner 29 April 2011 10:32:56PM 1 point [-]

How does your system handle jealousy, rage, and desire-for-revenge?