That's why I said "another human readable format". Of course the AGI can also answer in a non human readable format but that would defeat the point of the exercise.

If you ask a contemporary Chess computer that suddenly gets self awareness why it chooses one move over another it might tell you: Well given heuristics X and Y that's the outcome of my monte-carlo simulation, if you want here's the machine code and the whole stack trace.

For practical purposes that doesn't mean that the human understands the reasoning fully.