On IQ Accuracy:

As Yvain says, "people have been pretty quick to ridicule this survey's intelligence numbers as completely useless and impossible and so on" because if they're true, it means that the average LessWronger is gifted. Yvain added a few questions to the 2012 survey, including the ACT and SAT questions and the Myers-Briggs personality type question that I requested (I'll explain why this is interesting), and that give us a few other things to check against, which has made the figures more believable. The ridicule may be an example of the "virtuous doubt" that Luke warns about in Overconfident Pessimism, so it makes sense to "consider the opposite":

The distribution of Myers-Briggs personality types on LessWrong replicates the Mensa pattern. This is remarkable since the patterns of personality types here are, in many significant ways, the exact opposite of what you'd find in the regular population. For instance, the introverted rationalists and idealists are each about 1% of the population. Here, they are the majority and it's the artisans and guardians who are relegated to 1% or less of our population.

Mensa's personality test results were published in the December 1993 Mensa Bulletin. Their numbers.

So, if you believe that most of the people who took the survey lied about their IQ, you also need to believe all of the following:

• That most of these people also realized they needed to do IQ correlation research and fudge their SAT and ACT scores in order for their IQ lie to be believable.

• Some explanation as to why the average of lurker's IQ scores would come out so close to the average of poster's IQ scores. The lurkers don't have karma to show off, and there's no known incentive good enough to get so many lurkers to lie about their IQ score. Vaniver's figures.

• Some explanation for why the personality type pattern at LessWrong is radically different from the norm and yet very similar to the personality type pattern Mensa published and also matched my predictions. Even if they had knowledge of the Mensa personality test results and decided to fudge their personality type responses, too, they somehow managed to fudge them in such a way that their personality types accidentally matched my predictions.

• That they decided not to cheat when answering the Bayes birthday question even though they were dishonest enough to lie on the IQ question, motivated to look intelligent, and it takes a lot less effort to fudge the Bayes question than the intelligence and personality questions. (This was suggested by ArisKatsaris).

• That both posters and lurkers had some motive strong enough to justify spending 20+ minutes doing the IQ correlation research and fudging personality test questions while probably bored of ticking options after filling out most of a very long survey.

It's easier just to put the real number in the IQ box than do all that work to make it believable, and it's not like the liars are likely to get anything out of boasting anonymously, so the cost-benefit ratio is just not working in favor of the liar explanation.

If you think about it in terms of Occam's razor, what is the better explanation? That most people lied about their IQ, and fudged their SAT, ACT and personality type data to match, or that they're telling the truth?

Summary of criticism:

Possible Motive to Lie: The desire to be associated with a "gifted" group:

In re to this post, it was argued by NonComposMentis that a potential motive to lie is that if the outside world perceives LessWrong as gifted, then anyone having an account on LessWrong will look high-status. In rebuttal:

• I figure that lurkers would not be motivated to fudge their results because they don't have a bunch of karma on their account to show off and anybody can claim to read LessWrong, so fudging your IQ just to claim that the site you read is full of gifted people isn't likely to be motivating. I suggested that we compare the average IQs of lurkers and others. Vaniver did the math and they are very, very close..

• I argued, among other things, that it would be falling for a Pascal's mugging to believe that investing the extra time (probably at least $5 worth of time for most of us) into fudging the various different survey questions is likely to contribute to a secret conspiracy to inflate LessWrong's average IQ.

Did the majority avoid filling out intelligence related questions, letting the gifted skew the results?

Short answer: 74% of people answered at least one intelligence related question and since most people filled out only one or two, the fact that the self-report, ACT and SAT score averages are so similar is remarkable.

I realized, while reading Vaniver's post that if only 1/3 of the survey participants filled out the IQ score, this may have been due to something which could have skewed the results toward the gifted range, for instance, if more gifted people had been given IQ tests for schooling placement (and the others didn't post their IQ score because they did not know it) or if the amount of pride one has in their IQ score has a significant influence on whether one reported it.

So I went through the data and realized that most of the people who filled out the IQ test question did not fill out all the others. That means that 804 people (74% not 33%) answered at least one intelligence related question. As we have seen, the IQ correlations for the IQ, SAT and ACT questions were very close to each other (unsurprisingly, it looks like something's up with the internet test... removing those, it's 63% of survey participants that answered an intelligence related question). It's remarkable in and of itself that each category of test scores generated an average IQ so similar to the others considering that different people filled them out. I mean if 1/3 of the population filled out all of the questions, and the other 2/3 filled out none, we could say "maybe the 1/3 did IQ correlation research and fudged these" but if most of the population fills out one or two, and the averages for each category come out close to the averages for the other categories, why is that? How would that happen if they were fudging?

It does look to me like people gave whatever test scores they had and that not all the people had test scores to give but it does not look to me like a greater proportion of the gifted people provided an intelligence related survey answer. Instead it looks like most people provided an intelligence related survey answer and the average LessWronger is gifted.

Exploration of personality test fudging:

Erratio and I explored how likely it is that people could successfully fudge their personality tests and why they might do that.

• There are a lot of questions on the personality test that have an obvious intelligence component, so it's possible that people chose the answer they thought was most intelligent.

• There are also intelligence related questions where it's not clear which answer is most intelligent. I listed those.

• The intelligence questions would mostly influence the sensing/intuition dichotomy and the thinking/feeling dichotomy. This does not explain why the extraversion/introversion and perceiving/judging results were similar to Mensa's.