= 27d567bc2d48d9f40e9ccc20ea370b15)
cupholder comments on Undiscriminating Skepticism - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (1329)
You appear to be referring to Nisbett's paragraph starting with
A few sentences below that Nisbett refers to NAEP data to say that the reading score gap could be gone in 25 years and the science score gap in 75 years, if trends continue. [ETA: this is the 'largest study' that Nisbett cites. I'm sad Nisbett didn't give a more specific citation for it.]
The page you link appears to have data on the NAEP tests, but only for the mathematics tests. Clicking on the 'White-Black Gap' button, and then on the 'Age 17' tab (as Nisbett refers to 12th graders, so I am guessing that is what he and you are talking about...?) shows
The data linked do not appear to bear strongly on Nisbett's claims about the NAEP data (because Nisbett refers to the reading and science NAEP scores, not math), and I am also having difficulty seeing the 'small narrowing of the black/white gap between 1973 and 1982 and a fairly consistent gap thereafter.' in the data linked.
All in all, I am having difficulty substantiating your claim that Nisbett's claim is unsubstantiated by the data. I suspect either I am not interpreting your comment correctly, or the link in it happens to point to a data set other than the one you intended. Could you clarify?
(About the bigger question of whether black-white IQ differences have narrowed recently, it may be informative to read William Dickens and James Flynn's 2006 paper, which takes IQ test norming data and shows a narrowing of the IQ gap between 1972 and 2002. (Rushton and Jensen disagreed with the conclusions of that paper, but I find Dickens and Flynn's rebuttal convincing.)
I chose it at random and stopped with the first graph I found so nobody could accuse me of cherry picking. Looking more carefully at what Nisbett wrote, I see he did not specifically mention math scores.
I'm not sure if this makes a difference. If Nisbett was cherry-picking data, it doesn't really help his argument.
The one graph I looked at at random doesn't seem to support the claim that the gap (generally speaking) is narrowing and headed towards disappearing. Agreed?
When I see your random graph, I see the gap halving[!] from 1973 to 1990, widening through the 1990s, and maybe gradually shrunking since then. I see contradictory trends over the past 40 years, but it's more likely than not that the gap has resumed narrowing. So I'm not sure I do agree with you.
Since you write 'generally speaking' I guess you might be asking about the general trend as a whole from 1973 to now. I reckon that's an overall shrinking trend too.
To check my gut feeling more systematically, I did a quick regression of the score gap against year. (Not the best way to do it, but it beats eyeballing.) That gets me a .35 or .36 point shrinking per year depending on which assessment format I use for 2004. At that rate, the current gap (26 points in '08) would disappear in 70 to 75 years.
That's the same time period Nisbett gives for the disappearance of the science score gap, which I think is evidence against Nisbett 'cherry-picking' - if he cut out data because it had gaps that closed too slowly for his hypothesis, he would've left out the science data as well as the math data.
Summing up, I think I fundamentally disagree with you on the most likely interpretation of your graph.
Say what? The gap is 35 points in 1973 and 27 points in 1990. How is this halving?
Aha, I misunderstood which chart you had in mind. I thought that your link was intended to go to the data for 17 year olds, but that you were unable to link it directly because the page used Javascript to flip between the charts for different ages. I see now I'm wrong about that - one can link directly to the chart for each age, and it sounds like you were pointing to the age 9 data.
So I'll try this again with the 9 year olds. I've taken the liberty of looking at the black-white gap graph instead of the scale score graph so I don't have to do any mental arithmetic to get the gap size at each testing. Looks to me like the gap consistently narrowed from 1973 to 1986, and has fluctuated from 1986 so it's sometimes wider, sometimes thinner, but no overall trend since then.
Regressing gap size on year like I did before gives a shrinking of .24 or .25 points per year. So the picture is more mixed than for the older kids: there's an overall shrinking, but it's only two-thirds what you get for 17 year olds, and the trend looks like it's stalled since the late 80s.
Still, I am not sure that this means Nisbett is wrong. Looking at the bit of Nisbett you quote yourself downthread, Nisbett does not seem to say anything about the math scores, which means looking at the math scores would not tell us whether Nisbett is wrong or right.
It is possible that Nisbett cherry-picked by ignoring the math data, but I think a .25 point per year narrowing is still evidence against that idea. At a quarter point per year, the math gap would disappear in about a century, which isn't much longer than the 75 years Nisbett suggests for science.
Of course there are ways to interpret the graph to argue that the gap is narrowing and on track to disappear, but if you look at it and use your common sense, it's just not a reasonable conclusion.
The reasonable conclusion - as you allude to -- is that the gap has been pretty much stable for a number of years.
You put more trust in your common sense than I do. I try to avoid depending exclusively on what my common sense infers from eyeballing noisy time series - that way lies 'global warming stopped in 1998'esque error.
I find your preferred interpretation reasonable, but I don't see why it would be unreasonable to look at the entire data and see a net narrowing. (Especially if we lacked the 2008 data, as Nisbett did.)
If the choice is between trusting your common sense and trusting someone with an agenda, I would say go with your common sense.
Here's a thought experiment: You show the graph I linked to to 10 statisticians, except you replace the labels with something less politically charged. For example, the price of winter wheat versus the price of summer wheat. And you ask them to interpret the graph as far as long term trends go. I'm pretty confident that 10 out of 10 would interpret the graph the same way I did.
Ditto for global surface temperatures. Take the temperature label off the graph and tell people it's the dollar to yen exchange rate. I bet 10 out of 10 statisticians will say the rate is basically flat for the last 10 years.
cupholder has the empirical data - which, you will note, is increasing in all cases - but do you really imagine that no-one's tried a blind test?
No I do not imagine so. But I'm a little confused. Are you saying that the absence of significant cooling is the same thing as the presence of significant warming?
PS: The empirical data is not "increasing in all cases." Indeed, by most accounts global surface temperatures have not met or exceeded the high reached 12 years ago.
I am far less confident.
I bet it would depend on exactly which data set you gave them. Do you give them data for the past 10 years, data since 1998, the data since they started measuring temperatures with satellites as well as thermometers, or the longest-running data set, which runs from 1850 onwards? If you just give them the last decade of data, they might well just write it off as flat and noisy, but if you let them judge the recent numbers in the context of the entire time series, they might recognize them as flat-looking fuzz obscuring an ongoing linear trend.
That sounds nice, but I don't know how practical that would turn out to be, in this case or in general. In this particular case, how can I even tell with certainty whether you have 'an agenda' or not? And what if the key participants in a debate all have some agenda? It's very possible that Nisbett has a 'politically correct' (not that I like the phrase, but I can't think of a better way of putting it) agenda, and that Rushton and Jensen have a 'politically incorrect' agenda. How do I know, and what do I do if they do? And so on.
How can you tell anything with certainty? The fact is that you can't. Respectfully, it seems to me you are playing the "I'm such a skeptic" game.
Let me ask you this: Do you seriously doubt that Nisbett has an agenda?
I would give them the data since the 1970s when sattelite measurement became possible.
You're looking at age 9, cupholder is looking at 17.
In that case he is looking at the wrong graph when he talks about "your random graph."
Yes, but in that case you aren't looking at the data that Nisbett referred to. As cupholder pointed out
Agree, but as I said to cupholder, it doesn't help Nisbett's argument if he is cherry-picking data.
But Nisbett is quoting from a study "which found improvement on almost all tests for African American 12th graders". That study may not even have contained the data on 9-year-olds. You can ask "Why didn't that study include that data?", well because they were comparing data for 12th graders.
Actually, it's not clear to me what study he is talking about. Here's what he says:
So I went to the NAEP web site and looked at the very first graph I saw. What study do you think he is referring to?