Actually, I agree with you. The AI controls simulated tests. The GK controls the GK, regardless of what the AI's simulations say. I think the simulated tests rule only needs to be invoked if it's impractical to actually perform those tests. So, for example if someone did have friendliness criteria whose satisfaction could be proven with software and hardware available to use out-of-character, the simulated tests rule would not be invoked.

Kind of like in D&D you roleplay charisma checks, but roll the D20 for melee. At least the way I've been playing it.

For what it's worth, playing the AI, I would never argue from simulation because as a human I find such arguments spectacularly unconvincing (and there are plenty of humans who would deliberately do the opposite of what a simulation says they will just to show it who's boss). So the only way this would come up is if you for some reason asked me what my simulations predicted your response to X would be.

I do think my "GK's goal is to correctly identify friendly AI" makes the game way too easy for the AI. On the other hand, it's a useful thought experiment-- because if you can come up with a metric I can't create by waving my simulated tests wand, then we are on to something that might be worth $10.