Yes, that is the point. Psychologists often do tests where everything is held constant but for some particular feature of circumstance. Their purpose is to see what circumstantial features have what effects. The idea here is to build on that research to develop ways to test individual rationality.
(Epistemic) rationality has two major components:
Attending takes time, energy, quiet, etc. Circumstances where human rationality degrades include when:
It seems relatively easy to test rationality smarts; repeatedly give folks info and time to work new problems and measure their accuracy, calibration, etc. And I have an idea for testing for rationality toughness: compare performance on info-similar pairs of good/bad-circumstance problems.
For example, assume people are better at evaluating if a spouse is cheating when considering an acquaintance in their social circle, relative to a stranger or their own spouse. If so, we could pose them a pair of problems with very similar info structure, one with an easy spouse and one with a hard spouse. The closeness of their response in these two cases would then be a measure of their rationality toughness.
Of course this test may fail if the similarity is too obvious, or the pair are asked too closely in time. But maybe we don't even need to ask the same person the two questions; perhaps we could usefully compare someone's answer on a hard question to answers from a pool of similar people on matched easy questions.
While I haven't thought this through, it already suggests a training technique: consider matched hard/easy circumstance problems and compare your answers, separated by enough time that you forget most of your previous analysis.