lukeprog comments on Statistical Prediction Rules Out-Perform Expert Human Judgments - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (195)
Also, if I may be permitted to make a more general criticism in response to this post, I would say that while the article appears to be well-researched, it has demonstrated some of the worst problems I commonly notice on this forum. The same goes for the majority of the comments, even though many are knowledgeable and informative. What I have in mind is the fixation on concocting theories about human behavior and society based on various idées fixes and leitmotifs that are parts of the intellectual folklore here, while failing to notice issues suggested by basic common sense that are likely to be far more important.
Thus the poster notices that these models are not used in practice despite considerable evidence in their favor, and rushes to propose cognitive biases à la Kahneman & Tversky as the likely explanation. This without even stopping to think of two questions that just scream for attention. First, what is the importance of the fact that just about any issue of sorting out people is nowadays likely to be ideologically charged and legally dangerous? Second, what about the fact that these models are supposed to throw some high-status people out of work, and in a way that makes them look like they've been incompetent all along?
Regardless of whether various hypotheses based on these questions have any merit, the fact that someone could write a post without even giving them the slightest passing attention, offering instead a blinkered explanation involving the standard old LW/OB folklore, and still get upvoted to +40 is, in my opinion, indicative of some severe and widespread biases.
My intent was to summarize the literature on SPRs, not provide an account for why they are not used more widely. I almost didn't include that sentence at all. Surely, more analysis would be important to have in a post intending to discuss the psychological issues involved in our reaction to SPRs, but that was not my subject.
In pointing to cognitive biases as an explanation, I was merely repeating what Bishop & Trout & Dawes have suggested on the matter, not making up my own explanations in light of LW lore.
In fact, the arrows point the other way. Many of the authors cited in my article worked closely with people like Kahneman who are the original academic sources of much of LW lore.
Edit: I've added a clause about the source of the "cognitive biases" suggestion, in case others are tempted to make the same mistaken assumption as you made.