really it shouldn't be all that confusing. As agents, we're built with motivation systems, where out of all possible sensory patterns, some present to us as neutral, others as inherently desirable, and the last subset as inherently undesirable.
While I sympathize with you, I think you should decrease your threshold for apparent difficulty of problems.
For example, you should be able to choose between things that will make no sensory difference to you, such as the well-being of people in Xela. And of course you dodge the question of what is "enjoyable" - is a fistfight enjoyable if it makes you grin and your heart race but afterwards you never want to do it again? What algorithm should an AI follow to decide? You have to try and reduce "enjoyable" to things like "things you'd do again" or "things that make your brain release chemical cocktail X." And then you have to realize that those definitions are best met by meth, or an IV of chemical cocktail X, not by cool stuff like riding dinosaurs or having great sex.
While I sympathize with you, I think you should decrease your threshold for apparent difficulty of problems.
Along with what I just posted, I should also mention that I did say these two lines:
...at the most fundamental, there's nothing to the task of figuring out one's terminal values other than simply figuring out what sensory patterns are most 'enjoyable' in the most basic sort of way imaginable, on a timescale sufficiently long-term to be something one would be unlikely to refer to as 'akrasia'
It gets somewhat confusing when you factor in [...] akrasi
I don't know what my values are. I don't even know how to find out what my values are. But do I know something about how I (or an FAI) may be able to find out what my values are? Perhaps... and I've organized my answer to this question in the form of an "Outline of Possible Sources of Values". I hope it also serves as a summary of the major open problems in this area.
Using this outline, we can obtain a concise understanding of what many metaethical theories and FAI proposals are claiming/suggesting and how they differ from each other. For example, Nyan_Sandwich's "morality is awesome" thesis can be interpreted as the claim that the most important source of values is our intuitions about the desirability (awesomeness) of particular outcomes.
As another example, Aaron Swartz argued against "reflective equilibrium" by which he meant the claim that the valid sources of values are our object-level moral intuitions, and that correct moral reasoning consists of working back and forth between these intuitions until they reach coherence. His own position was that intuitions about moral principles are the only valid source of values and we should discount our intuitions about particular ethical situations.
A final example is Paul Christiano's "Indirect Normativity" proposal (n.b., "Indirect Normativity" was originally coined by Nick Bostrom to refer to an entire class of designs where the AI's values are defined "indirectly") for FAI, where an important source of values is the distribution of moral arguments the subject is likely to generate in a particular simulated environment and their responses to those arguments. Also, just about every meta-level question is left for the (simulated) subject to answer, except for the decision theory and ontology of the utility function that their values must finally be encoded in, which is fixed by the FAI designer.
I think the outline includes most of the ideas brought up in past LW discussions, or in moral philosophies that I'm familiar with. Please let me know if I left out anything important.