"Human interests" is meant in a vague sense -- there is some sense in which an agent that cures cancer (without doing anything else that humans would consider perverse or weird) is "more beneficial" than an agent that turns everything into paperclips, regardless of how you formalize things or deal with contradictions. This paper discusses technical problems that arise no matter how you formalize "human interests."

To answer the question that you clarify in later comments, I do not yet have even a vague satisfactory description. Formalizing "human interests" is far from a solved problem, but it's also not a very "technical" problem at present, which is why it's not discussed much in this agenda (though the end of section 4 points in that direction).