kalium comments on Advanced Placement exam cutoffs and superficial knowledge over deep knowledge - Less Wrong

4 Post author: JonahSinick 01 September 2013 09:01PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (18)

You are viewing a single comment's thread. Show more comments above.

Comment author: Ishaan 03 September 2013 06:26:42AM *  -1 points [-]

It's natural to ask what effect this has on incentives. Note that AP courses are taught to prepare students to get high marks on the AP exams, so that the grading scheme for AP exams propagates backward to the grading schemes for courses, which feed into college admissions. Taking many AP courses and performing just above the "5" level looks better than taking few AP courses and performing at the ~90% level.

Oh, so is that why schools use grades rather than percentages - an attempt to disincentive overspecialization and/or reduce pressure to overachieve? I've always wondered about that.

A more sane way to do this would be to give a percentage, but obfuscate it above a certain level. As in, John is at 78%, Wang is at 79%, but Ajeet, Alisha, Antwan are at 80%+ and the exact number shall not be revealed.

This keeps the disadvantage of undue emphasis on cut-off numbers (the 1% point that distinguishes C+ from B-) to a minimum while retaining the desired effect on incentives. There would still be a disadvantage at the single cut-off point (since 79% is significantly worse than 80+%) but at least the harm is contained around just one cut off point instead of several.

Comment author: kalium 05 September 2013 06:27:38PM 1 point [-]

Using percentages requires difficulty to be very precisely calibrated and requires you to throw out most of your range. If you make all your problems harder so I can now only do half rather than 100% in the allotted time, my grade should remain the same rather than going from an A to an F. Percentile is much more relevant than percentage, though it's not perfect either.

Comment author: Ishaan 05 September 2013 11:16:51PM *  -1 points [-]

Percentile is good when you want to identify where an individual is with respect to peers. You use it when you don't care about cohort effects and stuff like that. If I need 10 doctors, and I want the best doctors, then I can take the top 10 percentile in a class of 100. Even if they are all shitty MCAT scorers, I can't do without doctors so I've got to take the best of what I have. Likewise, even if they are all amazing scorers, I can't possibly take them all, so I'll just take the best.

Percentage is good when you want to identify an absolute level of skill. For example, if you wanted to see which students in a class of 100 were ready for algebra, you'd want to measure their absolute addition/subtraction/multiplication skills, not relative skills. If everyone is ready, then everyone can go on to Algebra. Likewise if no one is ready, they can all stay behind. I don't care where people are with respect to peers - I only care where they are.

I suppose the types of situations where you would attempt obfuscation would usually be of the "percentile" variety.