For those who might wish to defend the complexity-of-value thesis, what reasons do you have for thinking that human value is complex?
Humans have in their brains a lot of information about what morality is and is not, and it'd be silly to throw away all that information, especially considering that some of the simple underlying laws of morality might very well have some degree of context-sensitivity. Fundamental valuing of original contexts is one of the reasons I see that an FAI might bother to upload and run some humans or transhumans for a few septillions of years instead of only computing variations of convergent completely moral angelic computations instead.
In the Wiki article on complexity of value, Eliezer wrote:
But in light of Yvain's recent series of posts (i.e., if we consider our "actual" values to be the values we would endorse in reflective equilibrium, instead of our current apparent values), I don't see any particular reason, whether from evolutionary psychology or elsewhere, that they must be complex either. Most of our apparent values (which admittedly are complex) could easily be mere behavior, which we would discard after sufficient reflection.
For those who might wish to defend the complexity-of-value thesis, what reasons do you have for thinking that human value is complex? Is it from an intuition that we should translate as many of our behaviors into preferences as possible? If other people do not have a similar intuition, or perhaps even have a strong intuition that values should be simple (and therefore would be more willing to discard things that are on the fuzzy border between behaviors and values), could they think that their values are simple, without being wrong?