(Originally posted to my blog, The Rationalist Conspiracy; cross-posted here on request of Lukeprog.)
You’re the captain of a team, and you want to select really good players. How do you do it?
One way is through what I call positive selection. You devise a test – say, who can run the fastest – and pick the people who do best. If you want to be really strict, like if you’re selecting for the Olympics, you only pick the top fraction of a percent. If you’re a player, and you want to get selected, you have to train to do better on the test.
The opposite method is negative selection. Instead of one test to pick out winners, you design many tests to pick out losers. You test, say, who can’t run very well when it’s hot out, and get rid of them. Then you test who can’t run very well when it’s cold out, and get rid of them. Then you test running in the rain, and get rid of the losers there. And so on and so forth. When you’re strict with negative selection, you have lots and lots of tests, so that it’s very hard for any one person to pass through all the filters.
I think a big part of where American society’s gone wrong over the last hundred years is the ubiquitous use of negative selection over positive selection. (Athletics is one of the only exceptions. It’s apparently so important that people really care about performance – as opposed to, say, in medicine, where we exclude brilliant doctors if they don’t have the stamina to work ninety hours a week.) A single test can always be flawed; for example, IQ tests and SATs have many flaws. However, with negative selection, how badly you do is determined by the failure rate of every test combined. If you have twenty tests, and even one of them is so flawed it excludes good players, then your team will suck.
Elite college admissions is an example of a negative selection test. There’s no one way you can do really, really well, and thereby be admitted to Harvard. Instead, you have to pass a bunch of different selection filters: Are your SATs good enough? Are your grades good enough? Is your essay good enough? Are your extracurriculars good enough? Are your recommendations good enough? Failure on any one step usually means not getting admitted. And as competition has intensified, colleges have added more and more filters, like the supplemental applications top schools now require (in addition to the Common Application). It wasn’t always this way – Harvard used to admit primarily based on an entrance exam – until they discovered this let too many Jews in (no, seriously). More recently, the negative selection has been intensified by eliminating the SAT’s high ceiling.
Academia is another example of negative selection. To get tenure, first you have to get into a top PhD program. Then you have to graduate. Then you have to get a good recommendation from your advisor. Then you have to get a good postdoc. Then you have to get another good postdoc. Then you have to get a good assistant professorship. Then you have to get approved by the tenure committee. For the most part, if even one of those steps goes wrong – if you went to a second-tier PhD program, say – there’s no way to recover. Once you’re off the “track”, you’re off, and there’s no getting back on. It’s fail once, fail forever.
Grades are another example – A is a good grade, but there’s no excellent grade. There’s no grade that you only get if you’re in the top 0.1%. Hence, getting a really good GPA doesn’t mean excelling, so much as it means never failing. If you’re in high school and are taking six classes, if you fail one, your GPA is now 3.3 or less, regardless of how good you are otherwise.
In any field, at the top end, you tend to get a lot of variance. (Insert tales of the mad artist and mad mathematician.) Negative selection suppresses variance, by eliminating many of the dimensions on which people vary. Students at Yale are, for the most part, all strikingly similar – same socioeconomic class, same interests, same pursuits, same life goals, even the same style of dress. A lot of people tend to assume performance follows a bell curve, but in some cases, it’s more like a Pareto distribution: the top people do hundreds or thousands of times better than average. Hence, if you eliminate the small fraction of people at the very top, your performance is hosed. Fortunately for VC funds, the startup world is still positive selection.
Less obviously, a world with lots of negative selection might be a nasty one to live in. If you think of yourself as trying to eliminate bad, rather than encourage good, you start operating on the purity vs. contamination moral axis. Any tiny amount of bad, anywhere, must be gotten rid of, and that can lead to all sorts of nastiness. “When you are a Guardian of the Truth, all you can do is try to stave off the inevitable slide into entropy by zapping anything that departs from the Truth. If there’s some way to pump against entropy, generate new true beliefs along with a little waste heat, that same pump can keep the truth alive without secret police.”
Well, of course there's some kind of story that explains the failures, other than "he just wasn't good enough"; by assumption the guy wasn't let into the first graduate school off the street -- he had been a brilliant undergraduate. So something went wrong. The question is whether you should filter out a clearly capable person because something went wrong once. If you do, you're simply not optimizing for the right thing.
A rational deliberation on Candidate A would look like this: "This person obviously has issues, and he might not succeed here. But if he does, there's a good chance he'll succeed spectacularly -- at least, at a level above that of many students we're happy to call 'successful'. If he doesn't, well, some percentage of the blemish-free students we admit are going to fail anyway, so what's the difference?"
It is a fact of life that sometimes multiple iterations are required for something to work; but the school system does a poor job of accommodating this. Repeating a course as many times as necessary until you get an A is not something that the system encourages; if it were optimizing for the right things, it would be.
Yes, that was a flippant, stereotyped exaggeration -- especially since "humanities" includes philosophy and linguistics, where the mentality is often very similar to math. Still, like any good caricature, it has some basis in reality. Generally speaking, the "softer" the subject -- the fewer objective measures of competence -- the greater the reliance on pure status games, i.e. academic snobbery. And of course my whole point here is that this snobbery is present even in "hard" fields.
What I mean is that I'd want to hear the story. If the guy's wife died, that's one thing. If he got addicted to heroin, that's another. If I'm given no story, then I'll go with the undergrad. But I don't disagree with the rest of your assessment.
Fair enough.