I agree that if two things are indistinguishable in principle, it makes sense to use the same label for both.
It is not nearly as clear to me that "what makes me happy" and "what makes the world better" are indistinguishable sets as it seems to be to you, so I am not as comfortable using the same label for both sets as you seem to be.
You may be right that we don't use "happiness" to refer to the same things. I'm not really sure how to explore that further; what I use "happiness" to refer to is an experiential state I don't know how to convey more precisely without in effect simply listing synonyms. (And we're getting perilously close to "what if what I call 'red' is what you call 'green'?" territory, here.)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)
So it seems to me that the solution is use an expected utility function rather than a fixed utility function. Lets speak abstractly for the moment, and consider the space of all relevant utility functions (that is, all utility functions that would change the utility evaluate of an action). At each time step, we now will associate a probability of you transitioning from your current utility function to any of these other utility functions. For any given future state then, we can compute the expected utility. When you run your optimization algorithm to determine your action, what you therefore do is try and maximize the expected utility function, not the current utility function. So the key is going to wind up being assigning estimates to the probability of switching to any other utility function. Doing this in an entirely complete way is difficulty I'm sure, but my guess is that you can come to reasonable estimates that make it possible to do the reasoning.
I like this idea, but I would also, it seems, need to consider the (probabilistic) length of time each utility function would last.
That doesn't change your basic point, though, which seems reasonable.
The one question I have is this: In cases where I can choose whether or not to change my utility function - cases where I can choose to an extent the probability of a configuration appearing - couldn't I maximize expected utility by arranging for my most-likely utility function at any given time to match the most-likely universe at that time? It seems that would make life utterly pointless, but I don't have a rational basis for that - it's just a reflexive emotional response to the suggestion.