Oh. Well if we have enough examples that we don't need to define it, just create a few human-like AIs - don't worry about all that superintelligence nonsense, we can just create human-like AIs and run them faster. If we have enough insight into humans to be able to tell an AI how to predict them, it should be trivial to just skip the "tell an AI" part and predict what a human would come up with.

AI solved.

Or maybe you're hiding complexity behind definitions.