Because our conceptual framework is learned from experience, what we learn to seek is not necessarily what our genes "want". Of course if you place a human being in "the ancestral environment" then you will get learned values that serve the "aim of the genes" reasonably well - but not perfectly. In the modern environment, less so.
The idea of values coming from genes does not say anything about whether those desires are adaptive in the modern environment. Humans desire fat and sugar. Those desires are built in - coded in genes. That they are currently probably maladaptive is a different issue.
Saying that we have desires for chocolate gateau and ice cream that we must have learned from our environment seems like a "less helpful" way of looking at it the situation to me. It is better to regard chocolate gateau and ice cream as being learned associations with things actually valued. If they are to be classified as being "learned values", they are learned instrumental values.
Humans desire fat and sugar. Those desires are built in - coded in genes.
That's a half-truth, or maybe a truth-value-less sentence. One could just as easily say humans desire calories and vitamin C. Fat and sugar just happen to be, in the ancestral environment, means to these ends. Or perhaps humans simply desire survival and reproduction. I'm doubtful that any of these interpretations can claim to be the true one, at least until an individual human endorses one.
...It is better to regard chocolate gateau and ice cream as being learned associations
The jacket text for Keith Stanovich's The Robot's Rebellion sums up the book well:
The book is an excellent introduction to the first stage of Yudkowskian philosophy: We are robots in a mechanistic universe running on a swiss army knife of cognitive modules. But at least we finally noticed we're robots, and we can use the skills of rationality to hop off our habit treadmills and pursue our values instead. These values are complex and often arbitrary, but we can use our reflective capacities to extrapolate our values based on "higher-order" desires, a desire for preference consistency, and other considerations. All this is argued for at length in Stanovich's book. The only thing missing is a discussion of what to do about all this when AI arrives.