You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

atucker comments on [Altruist Support] How to determine your utility function - Less Wrong Discussion

7 Post author: Giles 01 May 2011 06:33AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (45)
Tags: instrumental rationality utility winning

Comments (45)

You are viewing a single comment's thread.

Comment author: atucker 01 May 2011 03:34:52PM *  0 points [-]

I like what you're trying to do and I hope you succeed, but I have some nitpicks.

Firstly, what the utility function is and what it's not. It belongs to the field of instrumental rationality, not epistemic rationality; it is not part of the territory. Don't expect it to correspond to something physical.

My utility function corresponds to specific structures in my brain.

That's it - you now have a shiny new utility function. And I want to help you optimize it. (Though it can grow and develop and change along with yourself; I want this to be a speculative process, not one in which you suddenly commit to an immutable life goal).

I don't think that it's that easy to figure out a utility function. There have been many things in my life that I've thought about and decided weren't that important, but then when I've actually tried them I discovered that I really really enjoy them. (Like, having a girlfriend, large numbers of friends, social hobbies, etc.)

You could say that that's a failure of my epistemic rationality, but I suspect that signalling is going to dominate hypotheticals that I think about but have never actually experienced. I think I would need a lot of practice in the inside view to be able to actually judge if I want something without trying it first.

Comment author: Giles 01 May 2011 11:16:16PM 0 points [-]

My utility function corresponds to specific structures in my brain.

Do you mean there's a particular structure in your brain sending out "good" and "bad" messages, and that's what you're trying to optimize? (i.e. that you would choose to wirehead).

Or do you mean in the sense that "utility functions have something to do with expressed and revealed preferences which necessarily correspond to structures in my brain?"

here have been many things in my life that I've thought about and decided weren't that important

Could they be covered under the umbrella of "I value things which I would enjoy if I tried"?

Comment author: atucker 02 May 2011 02:17:26AM *  0 points [-]

Or do you mean in the sense that "utility functions have something to do with expressed and revealed preferences which necessarily correspond to structures in my brain?"

More the latter.

Could they be covered under the umbrella of "I value things which I would enjoy if I tried"?

Yes, but I don't know how to optimize for those things (other than being generally adventurous/curious), since I don't know if I would enjoy them.

Powered by Reddit Powered by Reddit