JonahSinick comments on Advanced Placement exam cutoffs and superficial knowledge over deep knowledge - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (18)
It depends how obvious it is that some of the answers are wrong.
Obvious, of course, is a two-place word.
One experience I had that highlighted the importance of background knowledge for me was playing Alexei's Calibration Game. I had literally no knowledge about the sports questions- after confirming that, I just clicked A every time, and had 50% accuracy. But one of the categories of questions was "which of these postmasters general of the US served first?", which I was able to get 60-70% accuracy on, just by replacing that with the question "which of these two American names is more old-timey?"
I doubt a recent Chinese immigrant to the US would be able to hit 60% accuracy with that approach, because they don't have that good a sense of what's "old-timey" in the US. (And that's with only two options!) I've seen a handful of prep tests for subjects I know very little about, and my guessing accuracy there is close to chance.
Another way to put this: with multiple choice tests, reversed stupidity is intelligence, and that's a failing of the test. But identifying stupidity is a skill that requires some expertise.
But the expertise might not be in the area that's ostensibly being tested.