NancyLebovitz comments on How not to move the goalposts - Less Wrong

4 Post author: HopeFox 12 June 2011 03:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (71)

You are viewing a single comment's thread. Show more comments above.

Comment author: NancyLebovitz 12 June 2011 10:58:12PM 1 point [-]

then a red weasel's high test score is more likely to be a random fluke than a blue weasel's equally high score.

How likely is this if the test involves writing programs that work?

Comment author: cousin_it 12 June 2011 11:10:26PM *  5 points [-]

If the test is not susceptible to flukes, then my argument doesn't work. That said, flukes aren't necessarily extreme outliers. The red weasel you hired is more likely to be a slightly-below-standard performer that was having an unusually lucid day when you interviewed it.

On the other hand, Wei's argument works even if the test has no flukes. Here's one way to reformulate it: your binary decision to hire or reject a weasel is not informed by gradations of skill above the cutoff point. If blue weasels are more likely than red ones to hit the extreme high notes of software design (that weren't on the test because then the test would reject pretty much everyone), you'll see that inequality among the weasels you hire too.