I don't think there should be one system of knowledge creation. It's okay to have various different system in our society that works with different incentives.
I think an organisation like CFAR, provided it's well funded is more likely to invent effective techniques for rational thinking then academic psychologists.
GiveWell is also an organisation that creates valuable knowledge. They incentivise nonprofits to do good studies that prove the effectiveness of the nonprofits.
a giant Bayesian network of all humanity's knowledge
I think there's room for a crowdsourced version of this that works like Wikipedia.
Having a science replacement grow out of Givewell would be pretty interesting. I'll bet they'd do a better job of prioritizing research areas than government agencies like the National Science Foundation.
I feel that a lot of what's in LW (written by Eliezer or others) should be in mainstream academia. Not necessarily the most controversial views (the insistence on the MW hypothesis, cryonics, the FAI ...), but a lot of the work on overcoming biases should be there, be criticized there and be improved there.
For example, a few debiasing methods and a more formal explanation of LW's peculiar solution to free will (and more, these are only examples).
I don't really get why LW's content isn't in mainstream academia to be honest.
I get that peer review is not the best (far from it, although it's still the best we have, and post-publication peer-review is also improving, see PubPeer), that some would too readily dismiss LW's content, but not all. Lots would play by the rules and provide genuine criticisms during peer-review (which will lead to the alteration of the content of course), along with criticisms post publication. This is in my opinion something that has to happen.
LW, Eliezer, etc, can't stay on the "crank" level, not playing by the rules, publishing books and no papers. Blogs are indeed faster and reach a bigger amount of people, but I'm not arguing for only publishing in academia. Blogs can (and should) continue.
Tell me what you think, as I seem to have missed something with this topic.