Jack comments on Averaging value systems is worse than choosing one - Less Wrong

5 Post author: PhilGoetz 29 April 2010 02:51AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (56)

You are viewing a single comment's thread. Show more comments above.

Comment author: PhilGoetz 29 April 2010 05:51:20PM *  0 points [-]

The internal conflict (IC) in a value system is the negative of the sum, over all pairs of nodes, of the product of the node values and the connection weight between them. This is an energy measure that we want to minimize.

Why?

Good question. Because I prefer a value system that's usually not self-contradictory over one that's usually self-contradictory. I can't convince you that this is good if you are a moral nihilist, which is a very popular position on LW and, I think, central to CEV. If all possible value systems are equally good, by all means, choose one that tells you to love or hate people based on their fingerprints, and kill your friends if they walk through a doorway backwards.

Empirically, value systems with high IC resemble conservative religious values, which take evolved human values, and then pile an arbitrary rule system on top of them which gives contradictory, hard-to-interpret results resulting in schizophrenic behavior that appears insane to observers from almost any other value system, causes great pain and stress to its practitioners, and often leads to bloody violent conflicts because of their low correlation with other value systems.

Comment author: Jack 30 April 2010 05:11:59PM 2 points [-]

I upvoted because the formalization is interesting and the observation of what happens when we average values is a good one. But I'm still far from convinced IC is really what we need to worry about.

Empirically, value systems with high IC resemble conservative religious values, which take evolved human values, and then pile an arbitrary rule system on top of them which gives contradictory, hard-to-interpret results resulting in schizophrenic behavior that appears insane to observers from almost any other value system, causes great pain and stress to its practitioners, and often leads to bloody violent conflicts because of their low correlation with other value systems.

I think all of this applies to my liberal values: arbitrary rule system on top of evolved values? Check. Appears insane to observers from almost any other value system? Check. Causes great pain and stress to its practitioners? Check. Bloody violent conflicts because of their low correlation with other value systems? Double check!

And I still like my liberal values!

Comment author: PhilGoetz 01 May 2010 03:12:01AM 0 points [-]

Good point. Maybe tribal ethics have the least internal conflict, since they may be closest to an equilibrium reached by evolution.