= 27d567bc2d48d9f40e9ccc20ea370b15)
CCC comments on 2012 Survey Results - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (640)
Look. People misremember (and remember the largest value, and so on) in the way most favourable to themselves. While mistakes can of course go in either direction, they don't actually go in either direction. If you ask men to report their penis size (quite literally), they over-estimate; if you ask them to measure, they still overestimate but not by as much. This sort of error is absolutely the norm in any surveys. More so here, as the calibration (on Bayes date of birth question at least) was comparatively very bad.
The situation is anything but symmetric, given that the results are rather far from the mean, on a Gaussian.
Furthermore, given the interest in self improvement, people here are likely to have tried to improve their test scores by practice, which would have considerably lower effect on iqtest.dk unless you practice specifically the Raven's matrices.
The low scores on iqtest.dk are particularly interesting in light that the scores on the latter are a result of better assignment of priors / processing of probabilities (as, fundamentally, one needs to pick the choice which results in simplest - highest probability - overall pattern. If one is overconfident about the pattern they see being the best, one's score is lowered, so poor calibration will hurt that test more).
I intuit that this is likely to be a popular view among sceptics, but I do not recall ever being presented with research that supports this by anyone. To avoid the lure of "undiscriminating scepticism", I am requesting to see the evidence of this.
I agree that, for numerous reasons, self-reported IQ scores, SAT scores, ACT scores and any other scores are likely to have some amount of error, and I think it's likely for the room for error to be pretty big. On that we agree.
An average thirty points higher than normal seems to me to be quite a lot more than "pretty big". That's the difference between an IQ in the normal range and an IQ large enough to qualify for every definition of gifted. To use your metaphor, that's like having a 6-incher and saying it's 12. I can see guys unconsciously saying it's 7 if it's 6, or maybe even 8. But I have a hard time believing that most of these people have let their imaginations run so far away with them as to accidentally believe that they're Mensa level gifted when they're average. I'd bet that there was a significant amount of error, but not an average of 30 points.
If you agree with those two, then whether we agree over all just depends on what specific belief we're each supporting.
I think these beliefs are supported:
The SAT, ACT, self-reported IQ and / or iqtest.dk scores found on the survey are not likely to be highly accurate.
Despite inaccuracies, it's very likely that the average LessWrong member has an IQ above average - in other words, I don't think that the scores reported on the survey are so inaccurate that I should believe that most LessWrongers actually have just an average IQ.
LessWrong is (considering a variety of pieces of evidence, not just the survey) likely to have more gifted people than you'd find by random chance.
Do we agree on those three beliefs?
If not, then please phrase the belief(s) you want to support.
Even if every self-reported IQ is exactly correct, the average of the self-reported IQ values can still be (and likely will still be) higher than the average of the readership's IQ values.
Consider two readers, Tom and Jim. Tom does an IQ test, and gets a result of 110. Jim does an IQ test, and gets a result of 90. Tom and Jim are both given the option to fill in a survey, which asks (among other questions) what their IQ is. Neither Tom nor Jim intend to lie.
However, Jim seems significantly more likely to decide not to participate; while Tom may decide to fill in the survey as a minor sort of showing off. This effect will skew the average upwards. Perhaps not 30 points upwards... but it's an additional source of bias, independent of any bias in individual reported values.
I remember looking into this when I looked at the survey data. There were only a handful of people who reported two-digit IQs, which is consistent with both the concealment hypothesis and the high average intelligence hypothesis. If you assume that nonresponders have an IQ of 100 on average the average IQ across everyone drops down to 112. (I think this is assumption is mostly useful for demonstrative purposes; I suspect that the prevalence of people with two-digit IQs on LW is lower than in the general population.)
(You could do some more complicated stuff if you had a functional form for concealment that you wanted to predict, but it's not obvious to me that IQs on LW actually follow a normal distribution, which would make it hard to separate out the oddities of concealment with the oddities of the LW population.)
Ah! Good point! Karma for you! Now I will think about whether there is a way to figure out the truth despite this.
Ideas?
Hmmm. Tricky.
Select a random sampling of people (such as by picking names from the phonebook). Ask each person whether they would like to fill in a survey which asks, among other things, for their IQ. If a sufficiently large, representative sample is taken, the average IQ of the sample is likely to be 100 (confirm if possible). Compare this to the average reported IQ, in order to get an idea of the size of the bias.
Select a random sampling of lesswrongers, and ask them for their IQs. If they all respond, this should cut out the self-selection bias (though the odds are that at least some of them won't respond, putting us back at square one).
It's probably also worth noting that this is a known problem in statistics which is not easy to compensate for.
There's also the selection effect of only getting answers from "people who , when asked, can actually name their IQ".