There is a decent talk "Big Fat Fiasco" (about an hour long) that explains what happened with nutrition science.
Some of the interesting parts are near the end of part 2/start of part 3:
Specifically senator McGovern dismissing complaints from scientists that there was not enough evidence that fat caused hear disease by saying:
Senators don't have the luxury that a research scientist has of waiting until every last shred of evidence is in.
A little latter the video mentions that at the time 90% of all funding for research on heart disease was provided by the US Government and American Heart Association. Thus once both said that fat causes heart disease it was nearly impossible for scientists who got conflicting reports to get funding.
Edited conclusion to make it clearer:
There are two problems here:
1) The attitude that this area is too important to wait for "every last shred of evidence" and thus we must go with science based upon weak evidence.
Where else have I heard this, which appears to be prevalent in climate science and pandemic medicine today.
2) Nearly all funding provided by a few large organizations that are thus subject to politics and group think.
This appears to be true in most sciences today.
As such, unfortunately, it appears the case nutrition science isn't just an isolated incident.
I think not waiting for every last shred of evidence is the root of Bayesian thinking and is also the justification for considering there will be a singularity and that there are existential risks that we can do something about now. Well before the last shred of evidence is even remotely in sight.
I discovered Less Wrong a few months ago (courtesy of Harry Potter and the Methods of Rationality), and I am extremely grateful to have such a thoughtful place to have discussions and to learn new things. But one of the more significant hazards I've become aware of is confirmation bias. And since I began coming here a lot, I find that I evaluate new information through a "Less Wrong Lens" which officially means "well researched and thought out" which is perfectly fine but also includes some new or reinforced biases.
I recently realized the extent of the problem while reading "In Defense of Food". The basic premise (or most relevant one) of the book is that while science may one day be able to determine what is healthy on a nutrient-by-nutrient basis and let us craft whatever artificial foods we want, we are not nearly at that point yet. Every few years the prevailing beliefs of the nutritionist and scientific communities change, people scurry to catch up, and regardless, since transition from a "traditional" to "scientific" diet, certain nutrition-related diseases have gotten more prevalent, not less.
His argument is that traditional diets have often had thousands of years to evolve to match the needs and available food sources of populations. So while the variables and interactions may be complex enough that we don't know why, until science DOES figure it out, individual people are better off sticking to the diets of the past. At the same time, corporations that are interested only in marketing as much food as quickly as possible benefit from constantly changing scientific attitudes. (Disclaimer: yes, I'm oversimplifying again, but my point isn't even necessarily about the merits of the book so bear with me).
By the end of the book, I did agree with his basic premises. I'm not sure his solution is the single best one, but it's a large step up from the diet that the average westerner is going to have. It wasn't explicitly anti-science, just attempting to be realistic about what science can realistically accomplish, and what unique pitfalls come from giving science (or more importantly, politicians and corporations on whom scientists are dependent for funding) the position of power that religion and other cultural institutions once had.
But the first half of the book does have a pretty obvious goal, not of discrediting science per se, but "taking the wind out of science's sails". And in the wake of reading "Methods of Rationality" it absolutely rankled me. I could feel my memetic immune system going into overdrive, looking for reasons not to believe whatever the man ended up having to say. I'm currently unsure whether that had to do with the way he was writing, if he did have his own ax to grind, or if it was purely my own biases coloring the words. But whatever his motivations, I'm grateful for having found the book at the time I did, because it taught me a lesson about my own mind. My answer isn't to say "oh, science is just as flawed as everything else now," but I think I'll be able to approach things from a more neutral perspective.
Now, my purpose of posting this is two-fold. One, is I'm simply curious if other people had read the book and had anything to say about it, one way or another. But the other is to ask: do you have any sources you turn to specifically to help broaden your horizon from the prevailing mindset at Less Wrong? Websites that are not "anti-rational" or "anti-science," that you'd still consider trustworthy sources of information, but that help to offset certain biases that you might have accumulated here?