Eliezer_Yudkowsky comments on You're Entitled to Arguments, But Not (That Particular) Proof - Less Wrong

57 Post author: Eliezer_Yudkowsky 15 February 2010 07:58AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (221)

You are viewing a single comment's thread. Show more comments above.

Comment author: SilasBarta 15 February 2010 04:45:42PM *  19 points [-]

I was wondering how long it would be until the AGW issue was directly broached on a top-level post. Here I will state my views on it.

First, I want to fend off the potential charge of motivated cognition. I have spent the better part of two years criticizing fellow "libertarians" for trivializing the issue, and especially for their rationalizations of "Screw the Bengalis" even when they condition on AGW being true. I don't have the links gathered in one place, but just look here and here, and linked discussions, for examples.

That said, here are the warning signs for me (this is just to summarize, will gather links later if necessary):

1) Failed predictions. Given the complexity of the topic, your models inevitably end up doing curve-fitting. (Contrary to a popular misconception, they do not go straight from "the equations they design planes from" to climate models.) That gives you significant leeway in fitting the data to your theory. To be scientific and therefore remove the ability of humans to bias the data, it is vital that model predictions be validated against real-world results. They've failed, badly: they predicted, by existing measures of "global temperature", that it would be much higher than it is now.

2) Anti-Bayesian methodology accepted as commonplace. As an example, regarding the "hide the decline" issue with the tree rings, here's what happened: Scientists want to know how hot it was millenia ago. Temperature records weren't kept then. So, they measure by proxies. One common proxy is believed to be tree rings. But tree rings don't match the time period in which we have the best data.

The correct procedure at this point is to either a) recognize that they aren't good proxies, or b) include them in toto as an outlier data point. Instead, what they do is to keep all the data points that support the theory, and throw out the rest, calling it a "divergence problem", and further, claim the remaining points as additional substantiation of the theory. Do I need to explain here what's wrong with that?

And yet the field completely lacks journals with articles criticizing this.

3) Error cascades. Despite the supposed independence of the datasets, they ultimately come from only a few interbred sources, and further data is tuned so that it matches these data sets. People are kept out of publication, specifically on the basis that their data contradicts the "correct" data.

Finally, you can't just argue, "The scientists believe AGW, I trust scientists, ergo, the evidence favors AGW." Science is a method, not a person. AGW is credible to the exent that there is Bayesian evidence for it, and to the extent scientists are following science and finding Bayesian evidence. The history of the field is a history of fitting the data to the theory and increasing pressure to make sure your data conforms to what the high-status people decreed is correct.

Again, if the field is cleansed and audited and the theory turns out to hold up and be a severe problem, I would love for CO2 emissions to finally have their damage priced in so that they're not wastefully done, and I pity the fools that demand Bengalis go and sue each emitter if they want compensation. But that's not where we are.

And I don't think it's logically rude to demand that the evidence adhere to the standard safeguards against human failings.

Comment author: Eliezer_Yudkowsky 16 February 2010 12:16:50AM 9 points [-]

http://www.overcomingbias.com/2009/11/its-news-on-academia-not-climate.html

Yup, this behavior has long been typical when academics form competing groups, whether the public hears about such groups or not. If you knew how academia worked, this news would not surprise you nor change your opinions on global warming.

People are crazy, the world is mad. Of course there's gross misbehavior by climate scientists, just like the rest of academia is malfunctioning. But the amount of scrutiny leveled on climate science is vastly greater than the amount of scrutiny leveled on, say, the dietary scientists who randomly made up the idea that saturated fat was bad for you; and the scrutiny really hasn't turned up anything that bad, just typical behavior by "working" scientists. So I doubt that this is one of the cases where the academic field is just grossly entirely wrong.

Comment author: komponisto 16 February 2010 05:13:50PM 11 points [-]

People are crazy, the world is mad.

It just occurred to me that this really needs to be the title of a short popular book on heuristics and biases.

Comment author: Eliezer_Yudkowsky 16 February 2010 05:54:25PM 7 points [-]

The book title had already occurred to me, but it shouldn't be the first book in the series.

Comment author: Christian_Szegedy 17 February 2010 07:31:03AM 0 points [-]
Comment author: SilasBarta 16 February 2010 04:32:04PM 3 points [-]

People are crazy, the world is mad. Of course there's gross misbehavior by climate scientists, just like the rest of academia is malfunctioning. But the amount of scrutiny leveled on climate science is vastly greater than the amount of scrutiny leveled on, say, the dietary scientists...

Yes, and I expect that if you put this much scrutiny on most fields, where they are well-protected from falsification, you'd find the same thing. Like you said, scientists aren't usually trained in the rationalist arts, and can keep bad ideas alive much longer than they should be.

But this doesn't mean we should just shrug it off as "just the way it works"; we should appropriately discount their evidence for having a less reliable truth-finding procedure if we're not already assuming as much.

Comment author: brazil84 17 February 2010 12:27:50AM 0 points [-]

Another difference is that climate scientists are deriving lots and lots of attention, funding, and prestige out of worldwide concern for global warming.

Comment author: SilasBarta 17 February 2010 03:28:41AM 2 points [-]

True -- they seem ignorant of the "politics is the mind-killer" phenomenon. A boring research field may yield reliable science -- but once huge sums of money start to depend on its findings, you have to spend proportionally more effort keeping out bias -- such as by making your findings impossible to fake (i.e. no black-box methods for filtering the raw data).

Which climate researchers failed at tremendously.

Comment author: timtyler 16 February 2010 09:04:15AM 3 points [-]

http://en.wikipedia.org/wiki/Saturated_fat#Saturated_fat_intake_and_disease_-_Claimed_associations

...doesn't look as though scientists were "randomly making things" up to me.

Comment author: JackChristopher 17 February 2010 03:37:26AM *  6 points [-]
Comment author: timtyler 17 February 2010 10:59:58AM *  -1 points [-]

I am not particularly interested in a discussion of the virtues of saturated fat. It certainly seems like a bad example of scientists randomly making things up, though.

FWIW, here is a reasonably well-balanced analyisis of the 2010 study you mentioned:

"Study fails to link saturated fat, heart disease"

http://www.reuters.com/article/idUSTRE61341020100204

If you look at guidance on saturated fat it often recommends replacing it with better fats - e.g.:

"You should replace foods high in saturated fats with foods high in monounsaturated and/or polyunsaturated fats."

Epidemiological studies no-doubt include many who substituted saturated fats with twinkies.

Comment author: Eliezer_Yudkowsky 20 February 2010 03:11:09PM 3 points [-]

Where does the "guidance" come from? You can't cite "guidance" as evidence against the proposition that dietary scientists were making stuff up.

Comment author: timtyler 20 February 2010 03:31:59PM *  0 points [-]

I was explaining a problem with studies like the one cited - in exploring the hypotheses that saturated fats are inferior to various other fats. Basically, they don't bear on those hypotheses.

In this particular case, the authors pretty clearly stated that: "More data are needed to elucidate whether CVD risks are likely to be influenced by the specific nutrients used to replace saturated fat."