It's a fair question, but I don't have a good example to give you, and constructing one would take more effort than I feel like putting into it. So, no, sorry.

That's unfortunate, because this strikes me as a very important issue. Even being able to measure one's own rationality would be very helpful, let alone that of others.

That said, what you seem to be saying is that P2 is capable of making decisions that aren't influenced by their emotions and desires (via "standby rationality mode") but does not in fact do so except when taking rationality tests, whereas P1 is capable of it and also does so in real life.

I'm not sure I would put it in terms of "making decisions" so much as "making judgments," but basically yes. Also, P1 does make rational judgments in real life but the level of rationality depends on what is at stake.

If I've understood that correctly, then I agree that no rationality test can distinguish P1 and P2's ability to make decisions that aren't influenced by their emotions and desires.

Well one idea is to look more directly at what is going on in the brain with some kind of imaging technique. Perhaps self-deception or result-oriented reasoning have a tell tale signature.

Also, perhaps this kind of irrationality is more cognitively demanding. To illustrate, suppose you are having a Socratic dialogue with someone who holds irrational belief X. Instead of simply laying out your argument, you ask the person whether he agrees with Proposition Y, where Proposition Y seems pretty obvious and indisputable. Our rational person might quickly and easily agree or disagree with Y. Whereas our irrational person needs to think more carefully about Y; decide whether it might undermine his position; and if it does, construct a rationalization for rejecting Y. This difference in thinking might be measured in terms of reaction times.