EY is talking about oracles which answer questions. I am just talking about prediction.

But yes you do have a point that building a powerful predictive AI is not completely trivial. But it's certainly possible. If you have infinite computing power, you can just run Solomonoff induction.

Realistically we will have to find good approximations, and this might require using agenty-AI. And if so we will have to do work on controlling that AI. I believe this is possible, because it's a simple domain with a well specified goal, and no output channels except a single number.

Anyway, the other AI judge isn't an important or necessary part of my idea. I just wanted to have a simple outside judge of solutions. You could make the judge internal, have the AI use it's own probability estimates to decide when to output a solution. It is essentially doing that already by trying to predict what the judge will say to it's plan. The judge is redundant.