Open Thread, May 26 - June 1, 2014

BarbaraB

Previous Open Thread: http://lesswrong.com/r/discussion/lw/k94/open_thread_may_19_25_2014/

You know the drill - If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one.

3. Open Threads should start on Monday, and end on Sunday.

4. Open Threads should be posted in Discussion, and not Main.

Previous Open Thread: http://lesswrong.com/r/discussion/lw/k94/open_thread_may_19_25_2014/

You know the drill - If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one.

3. Open Threads should start on Monday, and end on Sunday.

4. Open Threads should be posted in Discussion, and not Main.

This paper has some interesting information and proposes the idea that the tests are designed to be resistant to short-term test prep:

A principal justification for the use of the SAT I and ACT in the admissions process is that such tests are designed to be insensitive to the high school curriculum and to short- term test preparation. If short term preparatory activities prior to taking the SAT I or ACT can have the effect of significantly boosting the scores of students above those they would have received without the preparation, both the validity and reliability of the tests as indicators of college readiness might be called into question.

It also alludes to there being multiple studies that have failed to find large effect sizes from test-prep courses. However, I'm not sure I quite believe that test prep is that ineffective. I never did test-prep for the SAT because I got a good enough score my first time to get into the university I wanted, but I did use test-prep books for AP tests, a few times for classes that I didn't even take, and I got 5/5 on most of those. I'm sure those were intended to be "cram-resistant" too, but they clearly weren't.

It's possible that the confusion here on LW comes from the systemic bias of the majority of people here being of above-average intelligence, though I don't know what the mechanism for that would be. I'll just add that "I notice that I am confused" as well, so something that we believe must be false...

My vague recollection of the SAT and ACT was that they were designed to test your ability to perform certain cognitive tasks, but didn't test whether you had any particular knowledge. There may have been exceptions on grammar questions, but I think that in the math section, you were given any formulas you might need -- whereas AP tests test knowledge extensively.

There may be a point at which you've taken so many test SAT/ACTs that, for lack of a better analogy, you've strengthened those cognitive muscles enough for it to make a difference, but cramming seems like an obviously ineffective way to handle this. AP tests, on the other hand, have a limited scope of facts for you to produce on command.