I believe that values are complex because I observe the apparent values to be complex and because I haven't seen a simple value system which wouldn't contradict both actual human behaviour and expressed preferences in some respect.

our "actual" values to be the values we would endorse in reflective equilibrium

Are there reasons to think that there is a reflective equilibrium and that it is unique? Anyway, if I had to construct an equilibrium, I would try to make it as close to the unstable apparent values which I endorse now and since this system is complex, the approximation may be complex as well. It is certainly possible to define a simple equilibrious value system, but I would be wary to talk about human values then.

This is an old post, but I think I have something relevant to say:

I think that because Yvain's posts focus on the times when your "behaviors" and your "values" conflict and your behaviors end up sabotaging important values, it gives the impression that behaviors just aren't important to your conscious mind. I don't think that's entirely true. Our conscious mind sometimes works in harmony with our behaviors and sees them as terminal values.

I think a good post of Yvain's that helps illustrate my point is his essay on Wanting, Liking, and Approving. We can think of our reflective equilibrium values as "Approving" and our more primal and instinctive behaviors as "Liking" and "Wanting." I think that an essential part of CEV and AI design is that people should maintain a balance between doing things they Approve of and doing things they Like. A future where people only do things they approve of and never do anything they like seems dull and puritanical. By contrast, a future where people just do what they Like and never do anything they Approve of seems pointlessly hedonistic. (I don't think that pure Wanting that is separated from both Liking and Approving has any value, however)

Both Liking and Approving are essential parts of a fulfilled life. A world that maximizes one at the expense of the other is not a good world. And while a reflective equilibrium might be able to simplify what you Approve of, I don't think it can ever simplify what you Like.

ADDED: Thinking about the concept of reflective equilibrium, I think that it is more of a way to prioritize values and behaviors than a way to discard them. If we assign a behavior low value in a reflective equilibrium that doesn't mean the behavior has no moral value. It simply means that in cases where that behavior interferes with a value we approve of more strongly we may have to stop it in order to achieve that more important goal.

I think Yvain's main point is that sometimes our more egosyntonic goals interfere with our egodystonic ones, and that in a reflective equilibrium we would assign priority to our most egosyntonic goals. I don't think that egosyntonic and egodystonic are binary categories, however. I think they are a continuum. I have many goals that are not inherently egodystonic, but become so when they are chosen as an alternative to another, even more important goal. Sometimes we might need to suppress one of our less egosyntonic behaviors to prevent it from interfering with an even more egosyntonic one. For instance, I don't disapprove of my reading funny websites on the Internet, I just approve of it less than I approve of my writing a novel. But if I'd already gotten a good bit of work done on a novel, I'd stop not approving of reading funny websites. It would be bad to delete our less egosyntonic desires altogether, because someday in the future we might find a way to satisfy both of those goals at once.

Also, after this recent exchange I had, I'm also less sure than before that Yvain is 100% right that egodystonic, approving goals should always trump "liking" goals. I can think of many behaviors I want to suppress so I can engage more frequently in doing things I approve of. But I also remember times in the past when I decided, after some reflection, that the approving part of my mind was a huge prude and I should have more fun. Maybe the conflict between the facets of our brain isn't as clear cut as I thought.

Again, Yvain previously argued that what made something a value and not a behavior was that it was egosyntonic. So we can answer the question of whether or not value is complex by asking if many of our complex behaviors are also egosyntonic. I know that in my case they definitely are. And even when they aren't the reason they aren't is often because they are obstructing an even more egosyntonic desire, not because I don't approve of them at all.

Is it from an intuition that we should translate as many of our behaviors into preferences as possible?

I know that in my case, it's because many of these behaviors are positive experiences for me, and that it would be negative to reject them unless they seriously interfered with something else. Furthermore, many of these behaviors are things I value, I just value them a little less than other things, and wish they wouldn't keep subverting my efforts to accomplish those other things.

For those who might wish to defend the complexity-of-value thesis, what reasons do you have for thinking that human value is complex?

Humans have in their brains a lot of information about what morality is and is not, and it'd be silly to throw away all that information, especially considering that some of the simple underlying laws of morality might very well have some degree of context-sensitivity. Fundamental valuing of original contexts is one of the reasons I see that an FAI might bother to upload and run some humans or transhumans for a few septillions of years instead of only computing variations of convergent completely moral angelic computations instead.

I think a good bit of my value complexity comes from the fact that I live in a human body.

Consider valuing tasty food.

What I will find "tasty" is based on a huge number of factors, like amount and type of fat, salt, sugar, texture, temperature, weather, what I've eaten that day, what I've eaten that week, what else I'm eating, time of day, the chemical state of my body, culture, etc.

And as long as I live in this body, I think that I won't be able to self-modify enough to eliminate most of this complexity in what I like (and all else being equal, want).

You can have high complexity if you value more than two things (like we do) by having high-entropy exchange rates. And as evolved creatures, entropy isn't just a good idea, it's the law. :P

So pure description length is all that interesting. More interesting is "model complexity" - that is, if you drew human value as a graph of connections and didn't worry about the connection strengths, what would the complexity of the graph be? Also interesting is "ideal outcome complexity" - would the ideal universe be tiled with lots of copies of something fairly simple, or would it be complicated?

I think that ideal outcome complexity is pretty definitely high, based on introspection, fiction, the lack of one simple widely known goal, evolution generating lots of separate drives with nonlinear responses, etc. But this only implies high model complexity if I don't value something that correlates with complexity - which is possible. So I'll have to think about that one a bit.

Most of our apparent values (which admittedly are complex) could easily be mere behavior [...] and therefore would be more willing to discard things that are on the fuzzy border between behaviors and values

Comparing "behaviour" and "values" as though they are the same sort of thing seems pretty strange.

It is probably best to think of those as different types of thing - and then there is no "fuzzy border".