shokwave comments on Statistical Prediction Rules Out-Perform Expert Human Judgments - Less Wrong

68 Post author: lukeprog 18 January 2011 03:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (195)

You are viewing a single comment's thread. Show more comments above.

Comment author: shokwave 18 January 2011 06:31:44AM *  2 points [-]

Keep in mind this is in the conclusion of lukeprog's post:

When there exists a reliable statistical prediction rule for the problem you're considering

Now,

But the notion 'works better' lacks a rigorous definition. Is the machine better if it wins 90% of the time by 5%, and fails the other 10% by 40%? It's not as simple as saying .9 * .05 > .1 * .4. The cost of error isn't necessarily linear.

If the cost of error isn't linear, determine what function it follows, then use that function instead of a linear function to compare the relative costs, which will tell you which works better.

Irrelevant is excessive.

I stand by it. The post is saying, given that SPRs work, work better than experts, and don't fail where experts don't, we should use them instead of experts. Your points were that SPRs don't always work, tend not to work in border cases, and might fail in dangerous cases. The first point is only true in cases this post is not concerned with, the second is equally true of experts and SPRs, and the third is also equally true of experts and SPRs.