"Claim: Reason can fit the world directly." - Here we say: "the map is not the territory"
"Claim: Language is neutral, and can fit the world directly." - Debunking this is a key part of Lakoff's research. He argues that the structure of language shapes our thoughts. I can't immediately think of a directly comparable line of thought on Less Wrong. ... but maybe someone else can see the connection more easily.
I don't know exactly what point Lakoff is trying to make, but there is an anti-Sequences point that I have been tempted to make from time to time. And Lakoff's words seem to be a pretty good jumping-off place.
I would say not only "The map is not the territory". I would say (with Lakoff?) that it is impossible to even speak about the territory with any precision. Language just doesn't work for that.
The foundation of Bayesian realism is the dogma that there is only one territory, though there can be many maps (one per map-maker). However, it is possible by use of language for two rational minds to agree on a map. Platonism in mathematics is the canonical example. Everyone agrees on what the standard model of arithmetic looks like. And there is agreement on standard 'models' of set theory as well; the only controversy deals with which model ought to be considered 'standard'. But, outside mathematics, although it is possible to reach agreement on what any particular map says, there is no way to reach agreement on which map best corresponds to the territory.
Paradoxically, it is the subjective 'maps' - things that exist only in people's heads - that are the cold, hard, clear-cut entities which can be studied using the mathematical tools of rationality. But it is the objective 'territories' which remain unknowable, controversial, and in some sense unspeakable.
There was a post a long time ago from Eliezer that I cannot find (edit: thank you Plasmon!) with a quick search of the site, where he had listed a set of characteristics ("blue", "is egg-shaped") and in the center a label for those characteristics ("blegg!"); and two graphs. One graph, which is the native description in most minds, has a node at the center, the label (blegg!) and nodes coming out (something is blue iff it is a bleg iff it is egg-shaped) and the other graph which had no label (something could be blue or egg-sh...
An article at The Edge has scientific experts in various fields give their favorite examples of theories that were wrong in their fields. Most relevantly to Less Wrong, many of those scientists discuss what their disciplines did that was wrong which resulted in the misconceptions. For example, Irene Pepperberg not surprisingly discusses the failure for scientists to appreciate avian intelligence. She emphasizes that this failure resulted from a combination of different factors, including the lack of appreciation that high level cognition could occur without the mammalian cortex, and that many early studies used pigeons which just aren't that bright.