Rationality requires intelligence, and the kind of intelligence that we use (for communication, progress, FAI, etc.) runs on language.
It seems that the place we should start is optimizing language for intelligence and rationality. One of SIAI's proposals includes using Lojban to interface between humans and an FAI. And of course, I should hope the programming language used to build a FAI would be "rational". But it would seem to me that the human-generated priors, correct epistemic rationality, decision theory, metaethics, etc. all depend on using a language that sufficiently rigorously maps to our territory.
Are "naturally evolved" languages such as English sufficient, with EY-style taboos and neologisms? Or are they sick to the core?
Please forgive and point me towards previous discussion or sequences about this topic.
It's pretty ridiculous that SIAI thinks it's a good idea to use Lojban to teach programs about ethics. The distinctive feature of Lojban is that it's easy to parse; but nowadays we have kinda decent natural language parsers. I used a constructed language to communicate with my class project in college, but only because I didn't want to bother with figuring out how to use an English parser, and because I knew no one but me would ever talk to it.
I'd bet a fair amount of money that the Lojban thing was Ben Goertzel's idea and a lot more that nobody at SIAI who is going to actually be doing FAI research thinks this is an even remotely good idea.