private_messaging comments on 2012 Survey Results - Less Wrong

80 Post author: Yvain 07 December 2012 09:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (640)

You are viewing a single comment's thread. Show more comments above.

Comment author: Epiphany 25 September 2013 05:05:49AM 0 points [-]

While mistakes can of course go in either direction, they don't actually go in either direction.

I intuit that this is likely to be a popular view among sceptics, but I do not recall ever being presented with research that supports this by anyone. To avoid the lure of "undiscriminating scepticism", I am requesting to see the evidence of this.

I agree that, for numerous reasons, self-reported IQ scores, SAT scores, ACT scores and any other scores are likely to have some amount of error, and I think it's likely for the room for error to be pretty big. On that we agree.

An average thirty points higher than normal seems to me to be quite a lot more than "pretty big". That's the difference between an IQ in the normal range and an IQ large enough to qualify for every definition of gifted. To use your metaphor, that's like having a 6-incher and saying it's 12. I can see guys unconsciously saying it's 7 if it's 6, or maybe even 8. But I have a hard time believing that most of these people have let their imaginations run so far away with them as to accidentally believe that they're Mensa level gifted when they're average. I'd bet that there was a significant amount of error, but not an average of 30 points.

If you agree with those two, then whether we agree over all just depends on what specific belief we're each supporting.

I think these beliefs are supported:

  • The SAT, ACT, self-reported IQ and / or iqtest.dk scores found on the survey are not likely to be highly accurate.

  • Despite inaccuracies, it's very likely that the average LessWrong member has an IQ above average - in other words, I don't think that the scores reported on the survey are so inaccurate that I should believe that most LessWrongers actually have just an average IQ.

  • LessWrong is (considering a variety of pieces of evidence, not just the survey) likely to have more gifted people than you'd find by random chance.

Do we agree on those three beliefs?

If not, then please phrase the belief(s) you want to support.

Comment author: private_messaging 26 September 2013 07:32:11PM *  2 points [-]

I intuit that this is likely to be a popular view among sceptics,

Sceptics in that case, I suppose, being anyone who actually does the most basic "Bayesian" reasoning, such as starting with a Gaussian prior when you should (and understanding how an imperfect correlation between self reported IQ and actual IQ would work on that prior, i.e. regression towards the mean when you are measuring by proxy). I picture there's a certain level of Dunning Kruger effect at play, whereby those least capable of probabilistic reasoning would think themselves most capable (further evidenced by calibration; even though the question may have been to blame, I'm pretty sure most people believed that a bad question couldn't have that much of an impact).

but I do not recall ever being presented with research that supports this by anyone.

Wikipedia to the rescue, given that a lot of stuff is behind the paywall...

http://en.wikipedia.org/wiki/Illusory_superiority#IQ

"The disparity between actual IQ and perceived IQ has also been noted between genders by British psychologist Adrian Furnham, in whose work there was a suggestion that, on average, men are more likely to overestimate their intelligence by 5 points, while women are more likely to underestimate their IQ by a similar margin."

and more amusingly

http://en.wikipedia.org/wiki/Human_penis_size#Erect_length

Do we agree on those three beliefs?

Just about any internet forum would select for people owning a computer and having an internet connection and thus cut off the poor, mentally disabled, and so on, improving the average. So when you state it this way - mere "above average" - it is a set of completely unremarkable beliefs.

It'd be interesting to check how common are advanced degrees among white Americans with actual IQ of 138 and above, but I can't find any info.

Comment author: Vaniver 26 September 2013 08:01:51PM *  2 points [-]

Sceptics in that case, I suppose, being anyone who actually does the most basic "Bayesian" reasoning, such as starting with a Gaussian prior when you should (and understanding how an imperfect correlation between self reported IQ and actual IQ would work on that prior, i.e. regression towards the mean when you are measuring by proxy).

This was one of the things I checked when I looked into the IQ results from the survey here and here. One of the things I thought was particularly interesting was that there was a positive correlation between self-reported IQ and iqtest.dk (which is still self-reported, and could have been lied on, but hopefully this is only deliberate lies, rather than fuzzy memory effects) among posters and a negative correlation among lurkers. This comment might also be interesting.

I endorse Epiphany's three potential explanations, and would quantify the last one: I strongly suspect the average IQ of LWers is at least one standard deviation above the norm. I would be skeptical of the claim that it's two standard deviations above the norm, given the data we have.

Comment author: private_messaging 26 September 2013 10:07:29PM 2 points [-]

and a negative correlation among lurkers

Wow, that's quite interesting - that's some serious Dunning-Kruger. Scatterplot could be of interest.

Thing to keep in mind is that even given a prior that errors can go either way equally, when you have obtained a result far from the mean, you must expect that errors (including systematic errors) were predominantly in that direction.

Other issue is that in a 1000 people, about 1 will have an IQ of >=146 or so , while something around 10 will have fairly severe narcissism (and this is not just your garden variety of overestimating oneself, but the level where it interferes with normal functioning).

Self reported IQ of 146 is thus not really a good sign overall. Interestingly some people do not understand that and go on how the others "punish" them for making poorly supported statements of exceptionality, while it is merely a matter of correct probabilistic reasoning.

The actual data is even worse than what comparisons of prevalence would suggest - 25% of people put themselves in the top 1% in some circumstances.

I endorse Epiphany's three potential explanations, and would quantify the last one: I strongly suspect the average IQ of LWers is at least one standard deviation above the norm. I would be skeptical of the claim that it's two standard deviations above the norm, given the data we have.

Yes, average of 115 would be possible.

Comment author: Vaniver 27 September 2013 12:05:25AM 2 points [-]

The actual data is linked in the post near the end. If you drop three of the lurkers- who self-reported 180, 162, and 156 but scored 102, 108, and 107- then the correlation is positive (but small). (Both samples look like trapezoids, which is kind of interesting, but might be explained by people using different standard deviations.)