brazil84 comments on Undiscriminating Skepticism - Less Wrong

97 Post author: Eliezer_Yudkowsky 14 March 2010 11:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (1329)

You are viewing a single comment's thread. Show more comments above.

Comment author: brazil84 18 March 2010 09:29:28AM 0 points [-]

Of course there are ways to interpret the graph to argue that the gap is narrowing and on track to disappear, but if you look at it and use your common sense, it's just not a reasonable conclusion.

The reasonable conclusion - as you allude to -- is that the gap has been pretty much stable for a number of years.

Comment author: cupholder 18 March 2010 11:28:48PM 2 points [-]

Of course there are ways to interpret the graph to argue that the gap is narrowing and on track to disappear, but if you look at it and use your common sense, it's just not a reasonable conclusion.

You put more trust in your common sense than I do. I try to avoid depending exclusively on what my common sense infers from eyeballing noisy time series - that way lies 'global warming stopped in 1998'esque error.

I find your preferred interpretation reasonable, but I don't see why it would be unreasonable to look at the entire data and see a net narrowing. (Especially if we lacked the 2008 data, as Nisbett did.)

Comment author: brazil84 19 March 2010 12:44:44AM -1 points [-]

If the choice is between trusting your common sense and trusting someone with an agenda, I would say go with your common sense.

Here's a thought experiment: You show the graph I linked to to 10 statisticians, except you replace the labels with something less politically charged. For example, the price of winter wheat versus the price of summer wheat. And you ask them to interpret the graph as far as long term trends go. I'm pretty confident that 10 out of 10 would interpret the graph the same way I did.

Ditto for global surface temperatures. Take the temperature label off the graph and tell people it's the dollar to yen exchange rate. I bet 10 out of 10 statisticians will say the rate is basically flat for the last 10 years.

Comment author: RobinZ 19 March 2010 03:23:30AM 4 points [-]

Ditto for global surface temperatures. Take the temperature label off the graph and tell people it's the dollar to yen exchange rate. I bet 10 out of 10 statisticians will say the rate is basically flat for the last 10 years.

cupholder has the empirical data - which, you will note, is increasing in all cases - but do you really imagine that no-one's tried a blind test?

Comment author: brazil84 19 March 2010 08:42:27AM *  -1 points [-]

cupholder has the empirical data - which, you will note, is increasing in all cases - but do you really imagine that no-one's tried a blind test?

No I do not imagine so. But I'm a little confused. Are you saying that the absence of significant cooling is the same thing as the presence of significant warming?

PS: The empirical data is not "increasing in all cases." Indeed, by most accounts global surface temperatures have not met or exceeded the high reached 12 years ago.

Comment author: RobinZ 19 March 2010 10:24:25AM -1 points [-]
  1. Every 10-year trendline in cupholder's data was increasing.

  2. If you give a statistician the 30-year or 130-year data set with the y-axis label taken off, they will tell you that there is no sign of a levelling-off.

Comment author: cupholder 19 March 2010 10:56:11PM *  2 points [-]

Every 10-year trendline in cupholder's data was increasing.

A quick clarification: for each of the data links I posted there, the trendline is calculated based on all of the data that's shown, i.e. for the post-1998 data the trendline is based on the last twelve years, for the post-1970s data the trendline is based on all of the post-1970s data, and so on. In other words, only the data for the last 10 years of data really have a 10-year trendline.

[ETA: Unless you mean you calculated 10-year trendlines for each data set yourself, in which case feel free to disregard this.]

Comment author: brazil84 19 March 2010 01:49:41PM -1 points [-]

Here's a plot of the UAH index from 1998 to 2009.

http://www.woodfortrees.org/plot/uah/from:1998/to:2009/plot/uah/from:1998/to:2009/trend

The linear trend is definitely decreasing for this particular plot.

If you give a statistician the 30-year or 130-year data set with the y-axis label taken off, they will tell you that there is no sign of a levelling-off.

I'm seriously skeptical of this.

P.S. Are you saying that the absence of significant cooling is the same thing as the presence of significant warming?

Comment author: wnoise 20 March 2010 05:39:44AM 0 points [-]

Note that changing the beginning data point to either 1997 or 1999 makes the regression line have a positive slope. It's not at all surprising that there is enough variability that cherry-picking data is possible. Stuffing a positive outlier at the beginning will, of course, tend to do this.

Comment author: brazil84 20 March 2010 10:28:57AM 0 points [-]

Note that changing the beginning data point to either 1997 or 1999 makes the regression line have a positive slope.

Agreed. I cherry-picked 1998 as a starting point to counter the claim that the data was increasing "in all cases."

Still, I would also note that as I explain on my blog post, there is some significance to the observation that global surface temperatures still have not exceeded the 1998 high. (According to the majority of leading temperature measurements.)

Comment author: RobinZ 19 March 2010 02:11:38PM 0 points [-]

Did you read the linked article?

In a blind test, the AP gave temperature data to four independent statisticians and asked them to look for trends, without telling them what the numbers represented. The experts found no true temperature declines over time.

"If you look at the data and sort of cherry-pick a micro-trend within a bigger trend, that technique is particularly suspect," said John Grego, a professor of statistics at the University of South Carolina.

[...]

The AP sent expert statisticians NOAA's year-to-year ground temperature changes over 130 years and the 30 years of satellite-measured temperatures preferred by skeptics and gathered by scientists at the University of Alabama in Huntsville.

Statisticians who analyzed the data found a distinct decades-long upward trend in the numbers, but could not find a significant drop in the past 10 years in either data set. The ups and downs during the last decade repeat random variability in data as far back as 1880.

Saying there's a downward trend since 1998 is not scientifically legitimate, said David Peterson, a retired Duke University statistics professor and one of those analyzing the numbers.

Identifying a downward trend is a case of "people coming at the data with preconceived notions," said Peterson, author of the book "Why Did They Do That? An Introduction to Forensic Decision Analysis."

1998 was a strong El Nino year - unusually high atmospheric temperatures that year in no way suggests that the earth has stopped heating.

Comment author: brazil84 19 March 2010 02:19:46PM *  -2 points [-]

Did you read the linked article?

Yes, and I'm not sure what your point is.

Are you claiming that the absence of a significant cooling trend is the same thing as the presence of a significant warming trend?

It's a very simple question. Why won't you answer it?

Incidentally, I wrote a blog post about the article in question which touches on these issues.

http://brazil84.wordpress.com/2009/10/27/more-on-global-cooling/

Comment author: RobinZ 19 March 2010 02:23:04PM 1 point [-]

I'm claiming that this one data set does not by itself support rejection of the body of theory that suggests global warming is occurring, and that it is intellectually dishonest to imply that it does.

Comment author: cupholder 19 March 2010 01:36:52AM *  2 points [-]

Here's a thought experiment: You show the graph I linked to to 10 statisticians, except you replace the labels with something less politically charged. For example, the price of winter wheat versus the price of summer wheat. And you ask them to interpret the graph as far as long term trends go. I'm pretty confident that 10 out of 10 would interpret the graph the same way I did.

I am far less confident.

Ditto for global surface temperatures. Take the temperature label off the graph and tell people it's the dollar to yen exchange rate. I bet 10 out of 10 statisticians will say the rate is basically flat for the last 10 years.

I bet it would depend on exactly which data set you gave them. Do you give them data for the past 10 years, data since 1998, the data since they started measuring temperatures with satellites as well as thermometers, or the longest-running data set, which runs from 1850 onwards? If you just give them the last decade of data, they might well just write it off as flat and noisy, but if you let them judge the recent numbers in the context of the entire time series, they might recognize them as flat-looking fuzz obscuring an ongoing linear trend.

If the choice is between trusting your common sense and trusting someone with an agenda, I would say go with your common sense.

That sounds nice, but I don't know how practical that would turn out to be, in this case or in general. In this particular case, how can I even tell with certainty whether you have 'an agenda' or not? And what if the key participants in a debate all have some agenda? It's very possible that Nisbett has a 'politically correct' (not that I like the phrase, but I can't think of a better way of putting it) agenda, and that Rushton and Jensen have a 'politically incorrect' agenda. How do I know, and what do I do if they do? And so on.

Comment author: brazil84 19 March 2010 08:53:31AM -1 points [-]

In this particular case, how can I even tell with certainty whether you have 'an agenda' or not?

How can you tell anything with certainty? The fact is that you can't. Respectfully, it seems to me you are playing the "I'm such a skeptic" game.

Let me ask you this: Do you seriously doubt that Nisbett has an agenda?

Do you give them data for the past 10 years, data since 1998, the data since they started measuring temperatures with satellites as well as thermometers, or the longest-running data set, which runs from 1850 onwards

I would give them the data since the 1970s when sattelite measurement became possible.

Comment author: cupholder 19 March 2010 10:42:29PM *  1 point [-]

How can you tell anything with certainty? The fact is that you can't. Respectfully, it seems to me you are playing the "I'm such a skeptic" game.

Sorry. I was being sloppy in my earlier comment, and using 'certainty' as a shorthand for 'certainty enough for me to label you as Having An Agenda, and therefore to reject your interpretation of the data as Tainted With An Agenda.' It is of course true that you can't tell anything inductive with cast-iron 100% certainty, but what I'm getting at is the question of how to get to what you or I would practically treat as certainty (like if I put a 95% probability on someone Having An Agenda).

Let me rephrase: in this particular case, how can I even tell whether you have 'an agenda' with sufficient certainty to disregard whatever you say about the data, and retreat to my own common sense gut feeling?

Let me ask you this: Do you seriously doubt that Nisbett has an agenda?

Do I doubt he has an agenda in the sense that he believes he's right? A tiny bit, but only in the sense that I am never completely sure of another person's motivation for stating something.

Do I doubt he has an agenda in the sense that he wants to convince other people of what he believes? Not really.

Do I doubt he has an agenda in the sense that he has an emotional investment in the argument as well as rational considerations? Only a little...but then again, who doesn't get emotionally invested in arguments?

Do I doubt he has an agenda in the sense that he has political motivations for his article as well as self-centered emotional and rational ones? Quite a lot, actually. I don't think I could reliably tell Nisbett's emotional motivations apart from those that spring from his political agenda (whatever that is - Nisbett sounds like a leftist to me, but how the hell do I really know? There were rightists who crapped on The Bell Curve too.) Does it even make sense to distinguish the two? I'm not sure. (I suddenly feel that these are good questions to think about. Thank you for prodding me into thinking of them.)

Also, for whatever it's worth, I am just as sure that Rushton and Jensen have 'an agenda,' however you want to define that, as Nisbett does. Do I throw all their papers out and just go with my common sense?

To clarify, this doesn't mean I can't get behind the idea of being alert to other people's biases on some subject, but I'm not willing to push that to the point of a dichotomy between my common sense vs. someone with an agenda. Taking the global warming example, I'm sure many climate scientists have 'an agenda,' but I'd still tend to accept their consensus interpretation of the data than my own common sense where the two differ, and I think that's reasonable if I don't have time to dig through all of the research myself.

I would give them the data since the 1970s when sattelite measurement became possible.

In that case I think I'm roughly 90% confident that fewer than '10 out of 10 statisticians will say the rate is basically flat for the last 10 years'. I am interpreting 'the rate is flat here' to mean that the net temperature trend is flat over time, as I believe we're talking about whether global warming is continuing and not whether global warming is accelerating. (Thought process here: I reckon a randomly selected statistician has at most a 4 in 5 chance of deciding that temperatures have been 'basically flat' for the last 10 years' based on the satellite data. Then the chance of 10 random statisticians all saying temperatures have been flat is 11%, so an 89% chance of at least one of them dissenting.)

Comment author: brazil84 20 March 2010 02:23:07AM *  1 point [-]

Do I doubt he has an agenda in the sense that he believes he's right?

By "having an agenda," I mean that Nisbett is emphasizing the facts that support a particular point of view and de-emphasizing the facts which undermine that point of view in order to persuade the reader.

So defined, one can ask whether Nisbett has an agenda. Do you have any doubt that Nisbett has an agenda?

I am interpreting 'the rate is flat here' to mean that the net temperature trend is flat over time,

So by your definition, the temperature trend is NOT basically flat between 1995 and the present, correct?

Comment author: cupholder 20 March 2010 05:43:15PM 1 point [-]

By "having an agenda," I mean that Nisbett is emphasizing the facts that support a particular point of view and de-emphasizing the facts which undermine that point of view in order to persuade the reader.

So defined, one can ask whether Nisbett has an agenda. Do you have any doubt that Nisbett has an agenda?

Not much. I think it is very likely that Nisbett suffers from confirmation bias about as much as everybody else.

So by your definition, the temperature trend is NOT basically flat between 1995 and the present, correct?

Eyeballing it I'd say it's much more likely that temperatures rose since 1995 than that they stayed flat, so I'd say you're pretty much correct. I wouldn't dogmatically say it's not flat in big capital letters, but I think the rising temperature hypothesis is a lot more likely than the flat temperature hypothesis.

I'd double check my intuition by running a regression, but that'd stack the deck because of autocorrelation, and I can't remember from the top of my head how to fit a linear model that accounts for that.

Comment author: brazil84 20 March 2010 06:04:42PM -2 points [-]

I think it is very likely that Nisbett suffers from confirmation bias about as much as everybody else.

I'm not sure what this means. Are you saying that every piece of written material has an agenda behind it as I've defined the term?

Eyeballing it I'd say it's much more likely that temperatures rose since 1995 than that they stayed flat, so I'd say you're pretty much correct

And do you agree that according to Phil Jones, there has been no statistically significant warming between 1995 and the present?

Comment author: ciphergoth 20 March 2010 07:09:34PM 0 points [-]

And do you agree that according to Phil Jones, there has been no statistically significant warming between 1995 and the present?

The fact that you quote this doesn't help your credibility. The Economist: Journalistic malpractice on global warming

Comment author: cupholder 20 March 2010 06:36:23PM *  -1 points [-]

I'm not sure what this means. Are you saying that every piece of written material has an agenda behind it as I've defined the term?

Not every piece of written material, but I'd bet that almost all lengthy pieces of writing intended to communicate a point to others have an agenda behind them, sure. There's always a temptation to round the numbers to your advantage, to leave out bits of data that might conflict with your hypothesis, to neglect to mention possible problems with your statistical tests, and so on.

Even ignoring that sort of thing, cognitive biases play an important role. Nisbett presumably had a half-formed opinion of the race and IQ argument even before he started researching it in depth. And that would in turn have affected which bits of relevant evidence got stuck in his mind. And that would in turn have hardened his opinion. You get positive feedbacks that push your opinion away from others that conflict with it. So even if Nisbett were consciously being as honest as possible, he could still be

emphasizing the facts that support a particular point of view and de-emphasizing the facts which undermine that point of view in order to persuade the reader

just because his mental database of facts is going to overrepresent the 1st kind of fact and underrepresent the 2nd - and precisely because of that, he is going to be sure that his point of view is obviously correct, and precisely because of that, he is going to be writing to persuade the reader of it - even though, as far as he knows, he is being completely honest!

(Tangent: it's somehow amusing and fitting that the person we're using to argue this point is the person whose most cited article is "Telling more than we can know: Verbal reports on mental processes.")

And do you agree that according to Phil Jones, there has been no statistically significant warming between 1995 and the present?

That's what he said.