Rationality requires intelligence, and the kind of intelligence that we use (for communication, progress, FAI, etc.) runs on language.
It seems that the place we should start is optimizing language for intelligence and rationality. One of SIAI's proposals includes using Lojban to interface between humans and an FAI. And of course, I should hope the programming language used to build a FAI would be "rational". But it would seem to me that the human-generated priors, correct epistemic rationality, decision theory, metaethics, etc. all depend on using a language that sufficiently rigorously maps to our territory.
Are "naturally evolved" languages such as English sufficient, with EY-style taboos and neologisms? Or are they sick to the core?
Please forgive and point me towards previous discussion or sequences about this topic.
A word language constructed from scratch based purely on what the creator thinks superior would indeed fall prey to your criticisms, but there a third possibility between a totally natural and totally artificial language. For lack of a better term, I'll call it a cultivated language. That is, a language built up out of real efforts to communicate for practical purposes, but with deliberate constraints imposed by the medium.
When language first formed, humans could mostly only communicate in a linear way, the linearity of communication using mouths and ears being the bottleneck. The introduction of writing systems could eventually have fixed this (through a visual non-linear language like saizai's), if not for inertia, as well as the fact that most non-intellectual people would be less interested in learning a language that had no carryover to speech.
But now we have the technology for a project that would place constraints on how people could communicate and just see what happens. In particular, if people could only communicate in 2D diagrams on a website designed for this language cultivation project, they might end up with something like saizai is trying to design, except it would be spontaneous.
And if there is any merit in Ian Ryan's arguments for a constructed language above, those insights could be incorporated into the constraints on the users to see how they play out. That seems to be the best of both worlds: a sort of guided evolution.