JonahSinick comments on Advanced Placement exam cutoffs and superficial knowledge over deep knowledge - Less Wrong

4 Post author: JonahSinick 01 September 2013 09:01PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (18)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vaniver 02 September 2013 09:39:17PM 2 points [-]

It depends how obvious it is that some of the answers are wrong.

Obvious, of course, is a two-place word.

One experience I had that highlighted the importance of background knowledge for me was playing Alexei's Calibration Game. I had literally no knowledge about the sports questions- after confirming that, I just clicked A every time, and had 50% accuracy. But one of the categories of questions was "which of these postmasters general of the US served first?", which I was able to get 60-70% accuracy on, just by replacing that with the question "which of these two American names is more old-timey?"

I doubt a recent Chinese immigrant to the US would be able to hit 60% accuracy with that approach, because they don't have that good a sense of what's "old-timey" in the US. (And that's with only two options!) I've seen a handful of prep tests for subjects I know very little about, and my guessing accuracy there is close to chance.

Another way to put this: with multiple choice tests, reversed stupidity is intelligence, and that's a failing of the test. But identifying stupidity is a skill that requires some expertise.

Comment author: JonahSinick 02 September 2013 09:47:10PM 4 points [-]

But identifying stupidity is a skill that requires some expertise

But the expertise might not be in the area that's ostensibly being tested.