Barry_Cotter comments on Statistical Prediction Rules Out-Perform Expert Human Judgments - Less Wrong

68 Post author: lukeprog 18 January 2011 03:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (195)

You are viewing a single comment's thread. Show more comments above.

Comment author: Barry_Cotter 19 January 2011 07:27:41PM 2 points [-]

This has a name in the literature, but I can't remember it at the moment

Goodhart's Law

A single SPR that took over for all the managers, or even a couple of very popular ones, would strongly encourage applicants to optimize for the variable most weighted in the equation.

W1(Quantitative skills) + W2(Written and Oral Communication Skills) + W3(Ability to work with loose supervision) + W4(Domain Expertise) + W5(Social Skills) + W6(Prestige Markers)

That said, I'd really like to see a wiki or other attempting-to-be-complete resource for finding an SPR for any particular application. Anyone got one?

No, but I imagine that taking a grab basket of plausible correlates of the desired trait and throwing them into a regression function would be a good first draft. Then iterate.