That's fair. How do you explain the in-person respondents? And more than that, the people who answered incorrectly and then were willing to bet money that they were right?
> 16 of the 22 Yale undergraduates who made the common error were sufficiently confident to bet on it, preferring to receive $2 for a correct response than $1 for sure.
That's a very large proportion! People who are rushing and who know that they are rushing should take the $1. I guess you could claim that that's not a large enough monetary motivation to induce truthful reflection but.....
If they're rushing then they can enter $5 as specifically instructed by the question. They can also check that if fits as a correct solution, which it does. The fact that so many people aren't doing that is interesting.
The report also details a study that tests instructions like "Be careful! Many people miss the following problem because they do not take the time to check their answer." Along with a big yellow warning icon. Does the warning work? Kindof! mTurk response times go from 22 seconds to 33 seconds. eLab responses go from 21 to 37 seconds (a...
They're not screwing up only decisions that don't matter, they're also screwing up decisions that don't matter to them. But many of those decisions matter quite a lot! The entire phenomenon of bike-shedding happens due to people focusing on the wrong things: things that they can trivially and lazily understand. Bike-shedding was first observed and described in the context of people designing nuclear power generating stations.
So in summary, some of the decisions people make are very important. Some of the people making those decisions are screwing them up because they are dumb, with rushing being a contributing but not exclusive factor, a factor that applies just as much in real life as it does in studies.