Ishaan comments on Advanced Placement exam cutoffs and superficial knowledge over deep knowledge - Less Wrong

4 Post author: JonahSinick 01 September 2013 09:01PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (18)

You are viewing a single comment's thread. Show more comments above.

Comment author: kalium 05 September 2013 06:27:38PM 1 point [-]

Using percentages requires difficulty to be very precisely calibrated and requires you to throw out most of your range. If you make all your problems harder so I can now only do half rather than 100% in the allotted time, my grade should remain the same rather than going from an A to an F. Percentile is much more relevant than percentage, though it's not perfect either.

Comment author: Ishaan 05 September 2013 11:16:51PM *  -1 points [-]

Percentile is good when you want to identify where an individual is with respect to peers. You use it when you don't care about cohort effects and stuff like that. If I need 10 doctors, and I want the best doctors, then I can take the top 10 percentile in a class of 100. Even if they are all shitty MCAT scorers, I can't do without doctors so I've got to take the best of what I have. Likewise, even if they are all amazing scorers, I can't possibly take them all, so I'll just take the best.

Percentage is good when you want to identify an absolute level of skill. For example, if you wanted to see which students in a class of 100 were ready for algebra, you'd want to measure their absolute addition/subtraction/multiplication skills, not relative skills. If everyone is ready, then everyone can go on to Algebra. Likewise if no one is ready, they can all stay behind. I don't care where people are with respect to peers - I only care where they are.

I suppose the types of situations where you would attempt obfuscation would usually be of the "percentile" variety.