Humans desire fat and sugar. Those desires are built in - coded in genes.
That's a half-truth, or maybe a truth-value-less sentence. One could just as easily say humans desire calories and vitamin C. Fat and sugar just happen to be, in the ancestral environment, means to these ends. Or perhaps humans simply desire survival and reproduction. I'm doubtful that any of these interpretations can claim to be the true one, at least until an individual human endorses one.
It is better to regard chocolate gateau and ice cream as being learned associations with things actually valued.
"Actually valued" suggests that ice cream is not actually valued except as a means to fat and sugar, which is definitely not true. Just try taking away someone's ice cream and offering lard and sugar in their stead.
Humans desire fat and sugar. Those desires are built in - coded in genes.
That's a half-truth, or maybe a truth-value-less sentence. One could just as easily say humans desire calories and vitamin C.
Calories, yes, vitamin C - probably not. It took quite a while for the link between vitamin C deficiency and the foods containing it to be discovered. Humans apparently don't have an instinctive craving for it - perhaps because their diet is normally saturated with it.
Or perhaps humans simply desire survival and reproduction.
Sure - e.g. the maternal in...
The jacket text for Keith Stanovich's The Robot's Rebellion sums up the book well:
The book is an excellent introduction to the first stage of Yudkowskian philosophy: We are robots in a mechanistic universe running on a swiss army knife of cognitive modules. But at least we finally noticed we're robots, and we can use the skills of rationality to hop off our habit treadmills and pursue our values instead. These values are complex and often arbitrary, but we can use our reflective capacities to extrapolate our values based on "higher-order" desires, a desire for preference consistency, and other considerations. All this is argued for at length in Stanovich's book. The only thing missing is a discussion of what to do about all this when AI arrives.