Perplexed comments on Statistical Prediction Rules Out-Perform Expert Human Judgments - Less Wrong

68 Post author: lukeprog 18 January 2011 03:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (195)

You are viewing a single comment's thread. Show more comments above.

Comment author: Perplexed 19 January 2011 03:53:30AM *  7 points [-]

that one is solid

Why do you say that? My reaction to that paper was very negative. In large part, it was the anecdotal flavor of the arguments made there, but also because I didn't see the two things I was specifically looking for:
* Citations of studies in which a linear model was constructed using one set of data, and then compared as to performance against the experts using a different set of data.
* Failing that, some numbers that would convince me that the failure to test models using different data than was used to construct them just doesn't matter.

Instead, here and in the 1996 study by Grove & Meehl, I find arguments from incredulity - in effect: "Do our critics really think that this really matters? Don't be absurd!". I also notice that this ideology is being promoted by a small number of researchers who repeatedly cite each other's work, and do not cite critics (except as strawmen).