It occurs to me that it would be very desirable to attempt to discover the CEV of humanity long before actually constructing an FAI to act under its direction.

Well, that brings us to a topic we have discussed before. Humans - like all other living systems - mosly act so as to increase entropy in their environment. That is http://originoflife.net/gods_utility_function/