Randaly comments on Statistical Prediction Rules Out-Perform Expert Human Judgments - Less Wrong

68 Post author: lukeprog 18 January 2011 03:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (195)
Tags: prediction reasoning expert_systems

Comments (195)

You are viewing a single comment's thread.

Comment author: Randaly 18 January 2011 05:13:48AM 0 points [-]

Your upload of Dawes's "The Robust Beauty of Improper Linear Models in Decision Making" seems to be broken- at least, I'm not able to access it.

Comment author: shokwave 18 January 2011 05:42:09AM 0 points [-]

at least, I'm not able to access it.

Neither.

Comment author: lukeprog 18 January 2011 05:50:41AM 0 points [-]

Dang. Fixed.

Comment author: shokwave 18 January 2011 07:41:17AM *  9 points [-]

Wow. I highly recommend reading the Dawes pdf, it's illuminating:

Expert doctors coded [variables from] biopsies of patients with Hodgkin's disease and then made an overall rating of the severity of the process. The overall rating did not predict the survival time of the 193 patients, all of whom died. (The correlations of survival time with ratings was virtually 0, some in the wrong direction). The variables that the doctors coded, however, did predict survival time when they were used in a multiple regression model.

In summary, proper linear models work for a very simple reason. People are good at picking out the right predictor variables ... People are bad at integrating information from diverse and incomparable sources. Proper linear models are good at such integration ...

He then goes on to show that improper linear models still beat human judgment. If your reaction to the top-level post wasn't endorsement of statistical methods for these problems, this pdf is a bunch more evidence that you can use to update your beliefs about statistical methods of prediction.

Comment author: TheOtherDave 18 January 2011 01:49:29PM 6 points [-]

People are good at picking out the right predictor variables ... People are bad at integrating information from diverse and incomparable sources.

That is a beautiful summary sentence, incidentally, and I am taking it with me as a shorthand "handle" for this whole idea.

I find it works well as a surface-level counter for the (alas, still inappropriately compelling) idea that a dumb algorithm can't get more accurate results than a smart observer.

Comment author: DSimon 18 January 2011 03:46:31PM 2 points [-]

Another possible metaphor is the pocket calculator.

It can find a number for any expression you can put into it, and in most cases it can do it way faster and more accurately than a human could. However, that doesn't make it a replacement for a human. An intelligent agent like a human is still needed for the crucial part of figuring out what expression would be meaningful to put into it.

Comment author: handoflixue 18 January 2011 09:49:10PM 0 points [-]

That is a very helpful metaphor for wrapping my head around both the advantages and limitations of SPR, thank you! :)

Powered by Reddit Powered by Reddit