Specifying that kind of thing (including specifying preference) is probably almost as hard as getting the AI's motivations right in the first place.

Though Paul Christiano had some suggestions along those lines, which (in my opinion) needed uploads (human minds instantiated in a computer) to have a hope of working...