In the Wiki article on complexity of value, Eliezer wrote:
The thesis that human values have high Kolmogorov complexity - our preferences, the things we care about, don't compress down to one simple rule, or a few simple rules.
[...]
Thou Art Godshatter describes the evolutionary psychology behind the complexity of human values - how they got to be complex, and why, given that origin, there is no reason in hindsight to expect them to be simple.
But in light of Yvain's recent series of posts (i.e., if we consider our "actual" values to be the values we would endorse in reflective equilibrium, instead of our current apparent values), I don't see any particular reason, whether from evolutionary psychology or elsewhere, that they must be complex either. Most of our apparent values (which admittedly are complex) could easily be mere behavior, which we would discard after sufficient reflection.
For those who might wish to defend the complexity-of-value thesis, what reasons do you have for thinking that human value is complex? Is it from an intuition that we should translate as many of our behaviors into preferences as possible? If other people do not have a similar intuition, or perhaps even have a strong intuition that values should be simple (and therefore would be more willing to discard things that are on the fuzzy border between behaviors and values), could they think that their values are simple, without being wrong?
This is an old post, but I think I have something relevant to say:
I think that because Yvain's posts focus on the times when your "behaviors" and your "values" conflict and your behaviors end up sabotaging important values, it gives the impression that behaviors just aren't important to your conscious mind. I don't think that's entirely true. Our conscious mind sometimes works in harmony with our behaviors and sees them as terminal values.
I think a good post of Yvain's that helps illustrate my point is his essay on Wanting, Liking, and Approving. We can think of our reflective equilibrium values as "Approving" and our more primal and instinctive behaviors as "Liking" and "Wanting." I think that an essential part of CEV and AI design is that people should maintain a balance between doing things they Approve of and doing things they Like. A future where people only do things they approve of and never do anything they like seems dull and puritanical. By contrast, a future where people just do what they Like and never do anything they Approve of seems pointlessly hedonistic. (I don't think that pure Wanting that is separated from both Liking and Approving has any value, however)
Both Liking and Approving are essential parts of a fulfilled life. A world that maximizes one at the expense of the other is not a good world. And while a reflective equilibrium might be able to simplify what you Approve of, I don't think it can ever simplify what you Like.
ADDED: Thinking about the concept of reflective equilibrium, I think that it is more of a way to prioritize values and behaviors than a way to discard them. If we assign a behavior low value in a reflective equilibrium that doesn't mean the behavior has no moral value. It simply means that in cases where that behavior interferes with a value we approve of more strongly we may have to stop it in order to achieve that more important goal.
I think Yvain's main point is that sometimes our more egosyntonic goals interfere with our egodystonic ones, and that in a reflective equilibrium we would assign priority to our most egosyntonic goals. I don't think that egosyntonic and egodystonic are binary categories, however. I think they are a continuum. I have many goals that are not inherently egodystonic, but become so when they are chosen as an alternative to another, even more important goal. Sometimes we might need to suppress one of our less egosyntonic behaviors to prevent it from interfering with an even more egosyntonic one. For instance, I don't disapprove of my reading funny websites on the Internet, I just approve of it less than I approve of my writing a novel. But if I'd already gotten a good bit of work done on a novel, I'd stop not approving of reading funny websites. It would be bad to delete our less egosyntonic desires altogether, because someday in the future we might find a way to satisfy both of those goals at once.
Also, after this recent exchange I had, I'm also less sure than before that Yvain is 100% right that egodystonic, approving goals should always trump "liking" goals. I can think of many behaviors I want to suppress so I can engage more frequently in doing things I approve of. But I also remember times in the past when I decided, after some reflection, that the approving part of my mind was a huge prude and I should have more fun. Maybe the conflict between the facets of our brain isn't as clear cut as I thought.
Again, Yvain previously argued that what made something a value and not a behavior was that it was egosyntonic. So we can answer the question of whether or not value is complex by asking if many of our complex behaviors are also egosyntonic. I know that in my case they definitely are. And even when they aren't the reason they aren't is often because they are obstructing an even more egosyntonic desire, not because I don't approve of them at all.
I know that in my case, it's because many of these behaviors are positive experiences for me, and that it would be negative to reject them unless they seriously interfered with something else. Furthermore, many of these behaviors are things I value, I just value them a little less than other things, and wish they wouldn't keep subverting my efforts to accomplish those other things.