Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: CCC 25 September 2013 07:29:03AM 5 points [-]

Even if every self-reported IQ is exactly correct, the average of the self-reported IQ values can still be (and likely will still be) higher than the average of the readership's IQ values.

Consider two readers, Tom and Jim. Tom does an IQ test, and gets a result of 110. Jim does an IQ test, and gets a result of 90. Tom and Jim are both given the option to fill in a survey, which asks (among other questions) what their IQ is. Neither Tom nor Jim intend to lie.

However, Jim seems significantly more likely to decide not to participate; while Tom may decide to fill in the survey as a minor sort of showing off. This effect will skew the average upwards. Perhaps not 30 points upwards... but it's an additional source of bias, independent of any bias in individual reported values.

In response to comment by CCC on 2012 Survey Results
Comment author: Epiphany 01 October 2013 09:03:51PM *  -1 points [-]

Ah! Good point! Karma for you! Now I will think about whether there is a way to figure out the truth despite this.

Ideas?

Comment author: private_messaging 23 September 2013 08:23:41AM *  0 points [-]

neither of which should be assumed to be likely to result in an average score that's 30 points too high, as mistakes like these could go in either direction

Look. People misremember (and remember the largest value, and so on) in the way most favourable to themselves. While mistakes can of course go in either direction, they don't actually go in either direction. If you ask men to report their penis size (quite literally), they over-estimate; if you ask them to measure, they still overestimate but not by as much. This sort of error is absolutely the norm in any surveys. More so here, as the calibration (on Bayes date of birth question at least) was comparatively very bad.

The situation is anything but symmetric, given that the results are rather far from the mean, on a Gaussian.

Furthermore, given the interest in self improvement, people here are likely to have tried to improve their test scores by practice, which would have considerably lower effect on iqtest.dk unless you practice specifically the Raven's matrices.

The low scores on iqtest.dk are particularly interesting in light that the scores on the latter are a result of better assignment of priors / processing of probabilities (as, fundamentally, one needs to pick the choice which results in simplest - highest probability - overall pattern. If one is overconfident about the pattern they see being the best, one's score is lowered, so poor calibration will hurt that test more).

Comment author: Epiphany 25 September 2013 05:05:49AM 0 points [-]

While mistakes can of course go in either direction, they don't actually go in either direction.

I intuit that this is likely to be a popular view among sceptics, but I do not recall ever being presented with research that supports this by anyone. To avoid the lure of "undiscriminating scepticism", I am requesting to see the evidence of this.

I agree that, for numerous reasons, self-reported IQ scores, SAT scores, ACT scores and any other scores are likely to have some amount of error, and I think it's likely for the room for error to be pretty big. On that we agree.

An average thirty points higher than normal seems to me to be quite a lot more than "pretty big". That's the difference between an IQ in the normal range and an IQ large enough to qualify for every definition of gifted. To use your metaphor, that's like having a 6-incher and saying it's 12. I can see guys unconsciously saying it's 7 if it's 6, or maybe even 8. But I have a hard time believing that most of these people have let their imaginations run so far away with them as to accidentally believe that they're Mensa level gifted when they're average. I'd bet that there was a significant amount of error, but not an average of 30 points.

If you agree with those two, then whether we agree over all just depends on what specific belief we're each supporting.

I think these beliefs are supported:

  • The SAT, ACT, self-reported IQ and / or iqtest.dk scores found on the survey are not likely to be highly accurate.

  • Despite inaccuracies, it's very likely that the average LessWrong member has an IQ above average - in other words, I don't think that the scores reported on the survey are so inaccurate that I should believe that most LessWrongers actually have just an average IQ.

  • LessWrong is (considering a variety of pieces of evidence, not just the survey) likely to have more gifted people than you'd find by random chance.

Do we agree on those three beliefs?

If not, then please phrase the belief(s) you want to support.

Comment author: ygert 10 September 2013 12:05:04PM 0 points [-]

You know, people do lie to themselves. It's a sad but true (and well known around here) fact about human psychology that humans have surprisingly bad models of themselves. It is simply true that if you asked a bunch of people selected at random about their (self-reported) IQ scores, you would get an average of more than 100. One would hope that LessWrongers are good enough at detecting bias in order to mostly dodge that bullet, but the evidence of whether or not we actually are that good at it is scarce at best.

In response to comment by ygert on 2012 Survey Results
Comment author: Epiphany 22 September 2013 01:30:14AM *  1 point [-]

Your unintentional lie explanation does not explain how the SAT scores ended up so closely synchronised to the IQ scores - as we know, one common sign of a lie is that the details do not add up. Synchronising one's SAT scores to the same level as one's IQ scores would most likely require conscious effort, making the discrepancy obvious to the LessWrong members who took the survey. If you would argue that they were likely to have chosen corresponding SAT scores in some way that did not require them to become consciously aware of discrepancies in order to synchronize the scores, how would you support the argument that they synched them on accident? If not, then would you support the argument that LessWrong members consciously lied about it?

Linda Silverman, a giftedness researcher, has observed that parents are actually pretty decent at assessing their child's intellectual abilities despite the obvious cause for bias.

"In this study, 84% of the children whose parents indicated that they fit three-fourths of the characteristics tested above 120 IQ. " (An unpublished study, unfortunately.)

http://www.gifteddevelopment.com/PDF_files/scalersrch.pdf

This isn't exactly the same as managing knowledge of one's own intellectual abilities, but if it would seem to you that parents would most likely be hideously biased when assessing their children's intellectual abilities even though, according to a giftedness researcher, this is probably not the case, then should you probably also consider that your concern that most LessWrong members are likely to subconsciously falsify their own IQ scores by a whopping 30 points (if that is your perception) may be far less likely to be a problem than you thought?

Comment author: private_messaging 10 September 2013 11:42:06AM *  1 point [-]

"Lie" is a strawman. One could report an estimate, mis-remember, report the other "IQ" (mental age / chronological age metric), or one may have took any one of entirely faulty online tests that report IQ as high to increase the referral rate (some are bad enough to produce >100 if the answers are filled in at random).

Comment author: Epiphany 22 September 2013 01:11:12AM *  0 points [-]

This would be a good point in the event that we were not discussing IQ scores generated by an IQ test selected by Yvain, which many people took at the same time as filling out the survey. This method (and timing) rules out problems due to relying on estimates alone, most of the potential for mis-remembering, (neither of which should be assumed to be likely to result in an average score that's 30 points too high, as mistakes like these could go in either direction), and, assuming that the IQ test Yvain selected was pretty good, it also rules out the problem of the test being seriously skewed. If you would like to continue this line of argument, one effective method of producing doubt would be to go to the specific IQ test in question, fill out all of the answers randomly, and report the IQ that it produces. If you want to generate a full-on update regarding those particular test results, complete with Yvain being likely to refrain from recommending this source during his next survey, write a script that fills out the test randomly and reports the results so that multiple people can run it and see for themselves what average IQ the test produces after a large number of trials. You may want to check to see whether Yvain or Gwern or someone has already done this before going to the trouble.

Also, there really were people whose concern it was that people were lying on the survey. Your "lie is a strawman" perception appears to have been formed due to not having read the (admittedly massive number of) comments on this.

Comment author: gwern 21 August 2013 04:07:23PM 5 points [-]

The cobbler's children don't always go unshod. :)

Comment author: Epiphany 22 August 2013 02:34:27AM *  0 points [-]

I did not intend to imply that you failed to back up your own data. That was intended as an amusing compliment.

Comment author: Epiphany 21 August 2013 08:29:43AM 5 points [-]

gwern.net

Comment author: Eugine_Nier 21 August 2013 06:09:51AM -1 points [-]

What I suspect is going on is that Epiphany is statistically innumerate (as suggested by her rather hilarious statement about 25% of Africans being above average), but doesn't want to loose status by admitting she doesn't understand the arguments.

Comment author: Epiphany 21 August 2013 08:21:07AM *  1 point [-]

Both of the citations I was given by you guys said clearly that they were uncertain about the connection between race and IQ. That is the reason I don't agree - because even your citations do not agree. I assume those are the best citations you have, so that your citations do not agree with you makes your belief look very bad indeed.

Also, by arguing that the reason I don't agree is because I am statistically innumerate and that the reason I don't agree is because I'm too inept to understand, you have made an ad hominem fallacy. Attacking the person does zilch to support your argument.

I can't believe I just saw an ad hominem attack on LessWrong. That is the the most obvious behavior that one avoids if one wants to have a rational debate.

Comment author: Eugine_Nier 21 August 2013 01:29:47AM 1 point [-]

Conversely, I can't think of any applications for which tying IQ to race is useful. Would you name three examples?

Well, seeing an unknown man approaching you at night, granted this is more about criminality than IQ but the correlation is the same.

Also thinking about whether affirmative action and the desperate impact doctrine are reasonable ideas.

Comment author: Epiphany 21 August 2013 04:30:17AM -2 points [-]

Well, seeing an unknown man approaching you at night

Actually, it is far more prudent to avoid a stranger approaching me at night, regardless of his race - depending on the environment I am in.

If he is approaching from a dark alley, I will head away from him, whatever his race. If he approaches me at a party full of friends, I will speak to him.

The crime statistics are not so incredibly different for blacks and whites that you can simply trust all of the whites.

Comment author: Lumifer 21 August 2013 04:12:17AM 2 points [-]

Sigh.

I don't believe you're listening and really have no inclination to play the "yes, but" game. Neither do I feel the need to prove anything to you.

You can believe whatever you want to believe, it's just that such an attitude looks strange here.

Comment author: Epiphany 21 August 2013 04:22:19AM -1 points [-]

You can believe whatever you want to believe, it's just that such an attitude looks strange here.

That is not my attitude. I have been asking you for research. Did you see what I discovered about "The Bell Curve"? What do you say about that?

Comment author: Lumifer 20 August 2013 06:10:33PM *  6 points [-]

"most published research is wrong"

Yes, I've read Ioannidis. However you're using this quote here as a rather blatant aid to your confirmation bias. There have been many, many studies which all show the same thing. These are findings which have been confirmed, re-confirmed, and confirmed once again.

Precisely because these results are so controversial they have been the subject of very thorough checking, vetting, and multiple attempts to debunk them. The results survived all this. What, do you think that for the last 50 years no one really tried to find holes in the studies showing racial IQ differences? Many highly qualified people tried. The results still stand.

Comment author: Epiphany 21 August 2013 03:55:36AM *  1 point [-]

Yes, I've read Ioannidis. However you're using this quote here as a rather blatant aid to your confirmation bias.

I think everyone should consider that published research findings are likely to be wrong each time they are seeking research findings. If you agree that we should be skeptical about research findings, why do you think that asking questions about whether the research controlled for multiple factors, was replicated etc. should be taken as evidence of confirmation bias? Maybe you disagree that we should be skeptical about research findings?

There have been many, many studies which all show the same thing.

Every single one? I would find that hard to believe for any topic, especially one as politically charged and controversial as this one, where both sides have a motive to bias research in their particular direction. If that is true, I would find it surprising. Assuming you were referring to the results of a meta-analysis, would you point to that meta-analysis please?

Precisely because these results are so controversial they have been the subject of very thorough checking, vetting, and multiple attempts to debunk them. The results survived all this. What, do you think that for the last 50 years no one really tried to find holes in the studies showing racial IQ differences? Many highly qualified people tried. The results still stand.

Are you saying that studies used for "The Bell Curve" did take into account the factors I mentioned, were replicated and / or may contain a meta-analysis that states that all the studies that could be found had similar findings?

If you aren't specific about what measures were taken to ensure quality in the information you're providing, I have no way to make the distinction between a matter of opinion and a matter of fact when you claim things like "The results survived all this." Please be specific about what particular quality features the data in The Bell Curve provides.

The Bell Curve

I started checking out this book because of your high praise and was surprised to find this:

On page 270, The Bell Curve clearly states: "The debate about whether and how much genes and environment have to do with ethnic differences remains unresolved"

Can you explain why you seem to be disagreeing with me, when both myself and The Bell Curve agree that we don't have a good way to tell whether IQ differences are nature or nurture? (Note: In addition to that, my view is also influenced by skepticism about research in general and an understanding that although IQ tests are correlated with various things, they have some limitations.)

View more: Next