This is an SQ because probably this was the first problem solved -it just so obivious - yet I cannot find it.

AFAIK it has not been solved and if it has, I would love to hear about it too. I also believe that, like you said, while it's possible for humans to negotiate and agree, any agreement would clearly be a compromise, and not the simultaneous fulfillment of everyone's different values.

CEV has, IIRC, a lot of handwaving and unfounded assumptions about the existence and qualities of the One True Utility Function it's trying to build. Is there something better?