It doesn't even test whether someone's doing math at the time. I could be doing all kinds of math and, in consequence, fail the exam.

I would say, rather, that tests generally have implicit preconditions in order for interpretations of their results to be valid.

Standing on a scale is a test for my weight that presumes various things: that I'm not carrying heavy stuff, that I'm not being pulled away from the scale by a significant force, etc. If those presumptions are false and I interpret the scale readings normally, I'll misjudge my weight. (Similarly, if I instead interpret the scale as a test of my mass, I'm assuming a 1g gravitational field, etc.)

Taking a math test in a classroom makes assumptions about my cognitive state -- that I'm awake, trying to pass the exam, can understand the instructions, don't have a gerbil in my pants, and so forth.