The lecturer in our Numerical cognition course told us of a result that went along the following lines. A schoolteacher was trying to teach his students to do basic arithmetic, and seeing them get the calculations wrong time after time. Then one day he decided to follow them out into town, where he saw that some of his students handled arithmetic just fine when they were doing grocery shopping, or working part-time selling things. Inspired, he returned to class and reworded his assignments to be about shopping, and guess what happened? The students failed just as miserably as they had before. The cognitive context was just too dissimiliar to the environment where they'd picked up the practice.
I got the impression that this wasn't just an isolated anecdote, but had also been replicated in more controlled studies. The reference he gave is to Jean Lave's Cognition in Practice - I have a copy of the book from the university library, but haven't had the time to read further yet. I'll see if I can skim it through this evening to find the part he was talking about.
This example helps clarify something for me. I don't think it's that the "cognitive context was...too dissimilar" for the students, I would guess that it's that they don't care in class. When they're doing they're job or shopping, they do care. But the obvious reply is: why do I hypothesize that cheating-examples make people care in a fictional context? Maybe someone can help say it clearly for me, but it just makes sense to me that math requires a higher threshold of "caring" than something like "cheating." If I were rea...
In their Evolutionary Psychology Primer, Cosmides and Tooby give an example of a hypothesized adaptation that allows us to detect cheaters in a certain type of logical task (Wason) that we generally fail at. In the Wason selection task (both article and wiki give examples) you are presented a type of logic puzzle that people tend to do poorly at and even formal training in logic helps little, yet when the examples involve cheating (such as "If you are to eat those cookies, then you must first fix your bed" and the task would be to figure out if someone whose eating the cookies did indeed fix the bed) perform much better (25% right in the regular task, 65-80% in this version, according to the article).
In the show The Wire, in season one, episode eight, Wallace, a teen-age drug dealer is asked by a young child to help her with her math homework. It's an addition and subtraction word problem about passengers on a bus (can't remember the numbers, but along the lines of, if the bus has 10 people on it and at the next stop 3 get on and 4 leave, etc.). Wallace rephrases the word problem to be about drugs and the kid gets it right. Wallace frustrated asks why and the kid replies along the lines of: "They beat you if you get the count wrong." (Edit:simpleton gives the quote as "Count be wrong, they fuck you up.")
C&T conclude that there are evolved "algorithms" in our brains that deal with social contract processing that explain why people do better on certain Wason selection tasks. The Wire points out a simpler possible explanation that their experiments did not control for: people do better on tasks they care about, unless one would like to suppose there are special math circuits in the brain for certain "social contract" situations.
Of course, I am not saying a fictional anecdote disproves C&T's claim, but it does point to something they didn't test for, and something that I find rather plausible.
Possible tests: Look at emotionally-motivating things that vary across culture and develop Wason selection tasks to test for that; look at various types of emotionally-motivating things (which I do not presume all emotional responses will affect the test results), and obviously, test The Wire example itself.