Well, with the tool, the problem as stated only requires the last of those. If an understanding of me is required in order to achieve a test condition, the capacity for it is assumed as long as the test condition is well-defined... right? I don't have to teach this thing the rules of Go or programming or me, as long as I can give it a test condition expressed concretely in terms of those things.

This is, admittedly, something of an abuse of your hypothetical.

Regardless, I agree that even just the last of those is a pretty tall order... as you say, not least because of the blind-optimization problem.

Then again, I was fairly dissatisfied with the Fun Theory sequence, so perhaps the exercise of putting together a set of measures that reflect my values nevertheless is an exercise worth doing, even without your tool (which does rather lower the stakes).