Comment on Pinpointing Utility

Sun, 03 Feb 2013 23:46:32 GMT

Unfortunately, it requires a bit of moral philosophy to pin down the relative weights of the utility functions. That is, the utility functions and their respective probabilities is not enough to uniquely identify the combined utility function.

Right, to get that answer you need to look inside your utility function... which you're uncertain about. Stated differently, your utility function tells you how to deal with uncertainty about your utility function, but that's another thing you're uncertain about. But luckily your utility function tells you how do deal with uncertainty about uncertainty about your utility function... I think you can see where this is going.

Naively, my intuition is that simply adding uncertainty about preferences as part of your ontology isn't enough because of this regress - you still don't even know in principle how to choose between actions without more precise knowledge of your utility function. However, this regress sounds suspiciously like the sort of thing that once formalized precisely isn't really a problem at all - just "take the limit" as it were.

Discuss