Yes. The problem is, if you look at the biggest disagreements humans have had - slavery, abortion, regional independence, whom to tax, how much the state should help people who can't help themselves, how much clothing women should wear, whether women should work outside the home - none of them can be resolved in this method. Religion, possibly; but only to the extent that a religion's followers care about the end goal of getting into heaven, and not to the extent that they have internalized its values.
I know Wei Dai has criticized CEV as a construct, I believe offering the alternative of rigorously specifying volition *before* making an AI. I couldn't find these posts/comments via a search, can anyone link me? Thanks.
There may be related top-level posts, but there is a good chance that what I am specifically thinking of was a comment-level conversation between Wei Dai and Vladimir Nesov.
Also feel free to use this thread to criticize CEV and to talk about other possible systems of volition.