I know Wei Dai has criticized CEV as a construct, I believe offering the alternative of rigorously specifying volition *before* making an AI. I couldn't find these posts/comments via a search, can anyone link me? Thanks.
There may be related top-level posts, but there is a good chance that what I am specifically thinking of was a comment-level conversation between Wei Dai and Vladimir Nesov.
Also feel free to use this thread to criticize CEV and to talk about other possible systems of volition.
Hmmm. That is not the scenario I was talking about. I was imagining that there would be a large number of people who would feel disenfranchised because their values were considered incoherent (or they were worried that their values might be thought incoherent). This coalition would seize political control of the CEV creation bureaucracy, change "coherent extrapolated" to "collective expressed" and then begin the negotiation process.
And? If you have multiple contradictory wishes what to do next, some of them are bound to be unfulfilled. CEV or negotiation are just ways to decide which ones.