billswift comments on The New Nostradamus - Less Wrong

13 Post author: Kaj_Sotala 12 September 2009 02:42PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (26)

You are viewing a single comment's thread. Show more comments above.

Comment author: billswift 12 September 2009 10:30:02PM *  4 points [-]

Tetlock says that it's very easy to write very simple computer programs that beat all of the people he assessed.

This is probably related to the fallacy of college admissions. Everyone in admissions thinks they can do better a job of predicting college success by using more criteria and their own professional judgement, and in every test using the SAT alone does better in aggregate.

Comment author: Alicorn 12 September 2009 10:54:42PM 4 points [-]

For more on this and related phenomena, see Michael Bishop, "In Praise of Epistemic Irresponsibility: How Lazy And Ignorant Can You Be?"

Comment author: roland 13 September 2009 06:46:27AM 0 points [-]

Great paper, I think it warrants a posting here on lw.

Comment author: Alicorn 13 September 2009 12:53:05PM 1 point [-]

I will probably have time to do this tomorrow or the day after. It would probably be a suitable topic.

Comment author: Psychohistorian 12 September 2009 11:29:35PM *  2 points [-]

in every test using the SAT alone does better in aggregate.

This is emphatically wrong. From the most pro-SAT source I can think of, SAT's + High School GPA are the most accurate predictor of first-year grades available. I admit the possibility that if you use a different measure of college success, my point is not valid.

It may be comforting to overstate the value of standardized testing, but that does not make it true, or excuse unfounded generalizations. One should be particularly careful to avoid such generalizations in a blue-green issue like the value of intelligence testing.

Comment author: roland 14 September 2009 05:05:38AM 2 points [-]

I think billswift's point was that the SAT is a better predictor as opposed to using human judgement as in reading the application letter, etc... It seems plausible that adding another "objective" measure like the HSGPA will further increase accuracy. But the big point is that using human judgement will DECREASE accuracy. The really interesting paper is the following as suggested by Alicorn:

Michael Bishop, "In Praise of Epistemic Irresponsibility: How Lazy And Ignorant Can You Be?"

Comment author: billswift 13 September 2009 02:09:19PM 0 points [-]

I don't remember my sources, but I came across this in several published books in the 1990s (I think they were from the late 1980s and early 90s but I'm not sure). This was before they weakened the SAT, so maybe that accounts for the difference, or maybe you're going by ad copy, rather than cognitive psych reports, which are attempting to show the value of the tests without offending the consumers of the tests.