To operationalize the situation, I would guess that the frequency with which mathematicians who have won famous prizes (Abel Prize, Fields Medal, etc.) would miss no questions at all (say, as 18 year olds) would be noticeably higher than the corresponding frequency for professors at top 50 math departments.

I agree that I expect it would be higher, though I would describe my expectation as "modest," which probably overlaps with "noticeable."