It probably says some mixture of bad things about academia (e.g. fear of looking silly) and bad things about LW (e.g. insufficient money to run randomized controlled trials, insufficient dedication to cite lots of related literature for every post the way lukeprog does).
Academia has flaws, like publication bias, closed access journals, expensive textbooks that don't allow comments, slower conversations, credentialism, lousy writing and maybe tenure. Less Wrong has flaws, like the fact that all its contributors are part-time, people think we're weird, it's not rewarding to write for, and maybe issues with the voting system. A blue sky kind of question to ask is how you might lay the foundation for something that outdoes both. What would academia look like if it was fully optimized for the internet age? (This question will look increasingly relevant if college enrollments continue declining and/or we start moving to a Coursera type online education model.)
One interesting thing to notice about academia is that it’s not monolithic the way Less Wrong is. You’ve got philosophers, sociologists, psychologists, economists, etc. each with their own somewhat disjunct body of knowledge. You could argue that they don’t talk to each other as much as they should, and that this is a problem. But I also see some advantages: mastering a single field is a more manageable job for a grad student than mastering many, and separately evolving bodies of thought might form a kind of system of checks against one another (e.g. if it weren’t for the psychologists, maybe economists would still be acting as though humans were perfectly rational agents).
Crazier proposals might have all the academics making their living off of a heavily subsidized prediction market or a giant Bayesian network of all humanity's knowledge that papers do updates on (screw confidence intervals). Having Scholarpedia cover the same range of ground as Wikipedia, but with cutting edge info and more rigor, is a tamer suggestion.
I don't think there should be one system of knowledge creation. It's okay to have various different system in our society that works with different incentives.
I think an organisation like CFAR, provided it's well funded is more likely to invent effective techniques for rational thinking then academic psychologists.
GiveWell is also an organisation that creates valuable knowledge. They incentivise nonprofits to do good studies that prove the effectiveness of the nonprofits.
a giant Bayesian network of all humanity's knowledge
I think there's room for a crowdsourced version of this that works like Wikipedia.
I feel that a lot of what's in LW (written by Eliezer or others) should be in mainstream academia. Not necessarily the most controversial views (the insistence on the MW hypothesis, cryonics, the FAI ...), but a lot of the work on overcoming biases should be there, be criticized there and be improved there.
For example, a few debiasing methods and a more formal explanation of LW's peculiar solution to free will (and more, these are only examples).
I don't really get why LW's content isn't in mainstream academia to be honest.
I get that peer review is not the best (far from it, although it's still the best we have, and post-publication peer-review is also improving, see PubPeer), that some would too readily dismiss LW's content, but not all. Lots would play by the rules and provide genuine criticisms during peer-review (which will lead to the alteration of the content of course), along with criticisms post publication. This is in my opinion something that has to happen.
LW, Eliezer, etc, can't stay on the "crank" level, not playing by the rules, publishing books and no papers. Blogs are indeed faster and reach a bigger amount of people, but I'm not arguing for only publishing in academia. Blogs can (and should) continue.
Tell me what you think, as I seem to have missed something with this topic.