Eliezer,

I am curious whether you are familiar with Aristotle's Nicomachean Ethics? Some of the discussion on fake utility functions, and the role of happiness in decision making seems to come into contact and/or conflict with the views Aristotle puts forward in that work. I'd be interested in knowing how your thoughts relate to those.

link to a free online copy of the Nicomachean Ethics: http://classics.mit.edu/Aristotle/nicomachaen.html