Comment on What's wrong with simplicity of value?

Wed, 27 Jul 2011 22:47:44 GMT

To borrow Eliezer's examples, do you think you would discard the value of boredom given enough time to reflect?

I don't know, but at least it seems plausible that I might. It's clear that I have boredom as an emotion and as a behavior, but I don't see a clear reason why I would want to make it into a preference. Certainly there are times when I wish I wouldn't get bored as easily as I actually do, so I don't want to translate boredom into a preference "as is".

If I think about why I might not want a future where everyone has no boredom and could enjoy the same thing over and over again, what's driving that seems to be an intuitive aversion towards triviality (i.e., things that look too easy or lack challenge). And if I think more about why I might not want a future where things look easy instead of difficult, I can't really think of anything except that it probably has to do with signaling that I'm someone who likes challenge, which does not seem like something I really want to base my "actual preferences" on.

Would you discard the desire to not be optimized too hard by an outside agent? Would you discard sympathy for other conscious humans?

These are similar for me. If I think about them long enough, it just becomes unclear why I might want to keep them.

Also, perhaps "keep" and "discard" aren't the right word here. What you actually need to do (in this view of what values are), is affirmatively create preferences (e.g., a utility function) from your intuitions. So for any potential value under consideration, you need reasons not for why you would want to "discard", but for why you would want to "create".

Discuss