Get data points on your current utility function via hypotheticals

1 Post author: Dorikka 24 April 2011 06:44PM

I've recently found that my utility function valued personal status and fame a whole lot more than I thought it did -- I previously had thought that it mostly relied on the consequences of my actions for other sentiences, but it turned out I was wrong. Obviously, this is a valuable insight -- I definitely want to know what my current utility function is; from there, I can decide whether I should change my actions or my utility function if the two aren't coordinated.

I did this by imagining how I would feel if I found out certain things. For example, how would I feel if everyone else was also trying to save the world? The emotional response I had was sort of a hollow feeling in the pit of my stomach, like I was a really mediocre being. This obviously wasn't a result of calculating that the marginal utility of my actions would be a whole lot lower in this hypothetical world (and so I should go do something else); instead, it was the fact that me trying to save the world didn't make me special any more -- I wouldn't stand out, in this sort of world.

(Epilogue: I decided that I hadn't done a good enough job programming my brain and am attempting to modify my utility function to rely on the world actually getting saved.)

Discussion: What other hypotheticals are useful?

Comments (4)

Comment author: AlexMennen 24 April 2011 10:46:47PM 6 points [-]

Happiness and utility are not the same thing. It's perfectly legitimate to say that you would feel better in situation A than you would in situation B, but would still prefer that situation B occurs, and this would mean that you assign higher utility to situation B.

Comment author: Oscar_Cunningham 24 April 2011 11:13:49PM 2 points [-]

This. If you find yourself considering trying to change your utility function, then something has gone badly wrong.

Comment author: Vladimir_Nesov 25 April 2011 05:37:12PM *  7 points [-]

I've recently found that my utility function valued [something]

This is silly. Just say that you found out that you value something. It's not like you have an explicit utility function module.

Comment author: Armok_GoB 24 April 2011 08:59:01PM 0 points [-]

You really need to make it clear what you are talking about, currently this sounds like a few paragraphs taken out of context from the middle of some larger article. At least post a link.