CronoDAS comments on Statistical Prediction Rules Out-Perform Expert Human Judgments - Less Wrong

68 Post author: lukeprog 18 January 2011 03:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (195)

You are viewing a single comment's thread.

Comment author: CronoDAS 18 January 2011 05:00:20AM 21 points [-]

An interesting story that I think I remember reading:

One study found that relatively inexperienced psychiatrists were more accurate at diagnosing mental illness than experienced ones. This is because inexperienced psychiatrists stuck closely to checklists rather than rely on their own judgment, and whether or not a diagnosis was considered "accurate" was based on how closely the reported symptoms matched the checklist. ;)

Comment author: Miller 18 January 2011 12:27:22PM 3 points [-]

If we are measuring the accuracy of A vs. B, we are implicitly measuring A against gold standard C, and B against gold standard C. If a better C is not readily available, we may choose to use A or B as an approximation, the choice of which determines our outcome.

Now I wonder:

Are the people that are sympathetic to the hypothesis that computers are better in the cases above (and ignored because of biases) assuming we made the fallacy of using humans as a gold standard?

Are the people that are sympathetic to the hypothesis that humans are better (and ignored because of biases) assuming we made the fallacy of using computers as a gold standard?

The union of which is a lot of upvotes. I can't decide which was meant.

Comment author: Miller 20 January 2011 07:47:00AM 1 point [-]

This is one of the top 3 rated comments on this post. I think you should specify more directly how this anecdote relates to how you interpret the article's intention.

Comment author: prase 20 January 2011 12:24:31PM 2 points [-]

He should specify where he has read that.

Comment author: CronoDAS 21 January 2011 01:34:18PM 0 points [-]

I don't remember. I may have actually heard one of my parents talking about it instead of reading it. So consider it an urban legend.