Eliezer_Yudkowsky comments on You're Entitled to Arguments, But Not (That Particular) Proof - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (221)
I was wondering how long it would be until the AGW issue was directly broached on a top-level post. Here I will state my views on it.
First, I want to fend off the potential charge of motivated cognition. I have spent the better part of two years criticizing fellow "libertarians" for trivializing the issue, and especially for their rationalizations of "Screw the Bengalis" even when they condition on AGW being true. I don't have the links gathered in one place, but just look here and here, and linked discussions, for examples.
That said, here are the warning signs for me (this is just to summarize, will gather links later if necessary):
1) Failed predictions. Given the complexity of the topic, your models inevitably end up doing curve-fitting. (Contrary to a popular misconception, they do not go straight from "the equations they design planes from" to climate models.) That gives you significant leeway in fitting the data to your theory. To be scientific and therefore remove the ability of humans to bias the data, it is vital that model predictions be validated against real-world results. They've failed, badly: they predicted, by existing measures of "global temperature", that it would be much higher than it is now.
2) Anti-Bayesian methodology accepted as commonplace. As an example, regarding the "hide the decline" issue with the tree rings, here's what happened: Scientists want to know how hot it was millenia ago. Temperature records weren't kept then. So, they measure by proxies. One common proxy is believed to be tree rings. But tree rings don't match the time period in which we have the best data.
The correct procedure at this point is to either a) recognize that they aren't good proxies, or b) include them in toto as an outlier data point. Instead, what they do is to keep all the data points that support the theory, and throw out the rest, calling it a "divergence problem", and further, claim the remaining points as additional substantiation of the theory. Do I need to explain here what's wrong with that?
And yet the field completely lacks journals with articles criticizing this.
3) Error cascades. Despite the supposed independence of the datasets, they ultimately come from only a few interbred sources, and further data is tuned so that it matches these data sets. People are kept out of publication, specifically on the basis that their data contradicts the "correct" data.
Finally, you can't just argue, "The scientists believe AGW, I trust scientists, ergo, the evidence favors AGW." Science is a method, not a person. AGW is credible to the exent that there is Bayesian evidence for it, and to the extent scientists are following science and finding Bayesian evidence. The history of the field is a history of fitting the data to the theory and increasing pressure to make sure your data conforms to what the high-status people decreed is correct.
Again, if the field is cleansed and audited and the theory turns out to hold up and be a severe problem, I would love for CO2 emissions to finally have their damage priced in so that they're not wastefully done, and I pity the fools that demand Bengalis go and sue each emitter if they want compensation. But that's not where we are.
And I don't think it's logically rude to demand that the evidence adhere to the standard safeguards against human failings.
http://www.overcomingbias.com/2009/11/its-news-on-academia-not-climate.html
People are crazy, the world is mad. Of course there's gross misbehavior by climate scientists, just like the rest of academia is malfunctioning. But the amount of scrutiny leveled on climate science is vastly greater than the amount of scrutiny leveled on, say, the dietary scientists who randomly made up the idea that saturated fat was bad for you; and the scrutiny really hasn't turned up anything that bad, just typical behavior by "working" scientists. So I doubt that this is one of the cases where the academic field is just grossly entirely wrong.
It just occurred to me that this really needs to be the title of a short popular book on heuristics and biases.
The book title had already occurred to me, but it shouldn't be the first book in the series.
A good related video:
http://www.ted.com/talks/sendhil_mullainathan.html
Yes, and I expect that if you put this much scrutiny on most fields, where they are well-protected from falsification, you'd find the same thing. Like you said, scientists aren't usually trained in the rationalist arts, and can keep bad ideas alive much longer than they should be.
But this doesn't mean we should just shrug it off as "just the way it works"; we should appropriately discount their evidence for having a less reliable truth-finding procedure if we're not already assuming as much.
Another difference is that climate scientists are deriving lots and lots of attention, funding, and prestige out of worldwide concern for global warming.
True -- they seem ignorant of the "politics is the mind-killer" phenomenon. A boring research field may yield reliable science -- but once huge sums of money start to depend on its findings, you have to spend proportionally more effort keeping out bias -- such as by making your findings impossible to fake (i.e. no black-box methods for filtering the raw data).
Which climate researchers failed at tremendously.
http://en.wikipedia.org/wiki/Saturated_fat#Saturated_fat_intake_and_disease_-_Claimed_associations
...doesn't look as though scientists were "randomly making things" up to me.
But what there saying fails to account for a lot of data. They're ignoring it.
A popular article (w/Seth Roberts) covering the issue: http://freetheanimal.com/2009/09/saturated-fat-intake-vs-heart-disease-stroke.html
2010 Harvard School of Public Health (intervention/meta-analysis): Meta-analysis of prospective cohort studies evaluating the association of saturated fat with cardiovascular disease
Saturated fat, carbohydrate, and cardiovascular disease
Another meta-analysis: The questionable role of saturated and polyunsaturated fatty acids in cardiovascular disease
Population Studies: Cardiovascular disease in the masai
Cholesterol, coconuts, and diet on Polynesian atolls: a natural experiment: the Pukapuka and Tokelau island studies
Cardiovascular event risk in relation to dietary fat intake in middle-aged individuals: data from The Malmö Diet and Cancer Study
I am not particularly interested in a discussion of the virtues of saturated fat. It certainly seems like a bad example of scientists randomly making things up, though.
FWIW, here is a reasonably well-balanced analyisis of the 2010 study you mentioned:
"Study fails to link saturated fat, heart disease"
http://www.reuters.com/article/idUSTRE61341020100204
If you look at guidance on saturated fat it often recommends replacing it with better fats - e.g.:
"You should replace foods high in saturated fats with foods high in monounsaturated and/or polyunsaturated fats."
Epidemiological studies no-doubt include many who substituted saturated fats with twinkies.
Where does the "guidance" come from? You can't cite "guidance" as evidence against the proposition that dietary scientists were making stuff up.
I was explaining a problem with studies like the one cited - in exploring the hypotheses that saturated fats are inferior to various other fats. Basically, they don't bear on those hypotheses.
In this particular case, the authors pretty clearly stated that: "More data are needed to elucidate whether CVD risks are likely to be influenced by the specific nutrients used to replace saturated fat."