Quoting Eliezer: "A minor, muddled preference of 60% of humanity might be countered by a strong, unmuddled preference of 10% of humanity." So any good Marxist will be able to imagine the rich and powerful getting their way in the computation of CEV just as they get their way today: by inducing muddle in the masses.

There's little reason for them to bother with such nonsense - if they are building and paying for the thing in the first place.

CEV may be a utilitarian's wet dream - but it will most-likely look like a crapshoot to the millionaires who are actually likely to be building machine intelligence.

The "I don't see how I could reasonably expect anything much better" seems likely to be a failure of the imagination.

And here I was considering it a victory of reason. :)

It seemed as though you were failing to forsee opposition to CEV-like schemes. There are implementation problems too - but even without those, such scenarios do not seem very likely to happen.