I've seen there's discussion on LW about rationality, namely, about what it means. I don't think a satisfactory answer can be found without defining what rationality is not. And this seems to be a problem. As far as I know, rationality on LW does not include systematic methods for categorizing and analyzing irrational things. Instead, the discussion seems to draw a circle around rationality. Everyone on LW is excepted to be inside this circle - think of it as a set in a Venn diagram. On the border of the circle there is a sign saying: "Here be dragons". And beyond the circle there is irrationality.
How can we differentiate the irrational from the rational, if we do not know what the irrational is?
But how can we approach the irrational, if we want to be rational?
It seems to me there is no way to give a satisfactory account of rationality from within rationality itself. If we presuppose rationality is the only way to attain justification, and then try to find justification for rationalism (the doctrine according to which we should strive for rationality), we are simply making a circular argument. We already presupposed rationalism before trying to find justification for doing so.
Therefore it seems to me we ought to make a metatheory of rationality in order to find out what is rational and what is irrational. The metatheory itself has to be as rational as possible. That would include having an analytically defined structure, which permits us to at least examine whether the metatheory is logically consistent or inconsistent. This would also allow us to also examine whether the metatheory is mathematically elegant, or whether the same thing could be expressed in a simpler form. The metatheory should also correspond with our actual observations so that we could figure out whether it contradicts empirical findings or not.
How much interest is there for such a metatheory?
Yudkowsky says:
A normative belief in rationality is, as far as I can tell, not possible for someone who does not have a clear concpetion of what rationality is. I am trying to present tools for forming such a conception. The theory I am presenting is, most accurately, a rationally constructed language, not a prescriptive theory on whether it is moral to be rational. The merit of this language is that it should allow you to converse about rationality with mysticists or religious people so that you both understand what you are talking about. It seems to me the ID vs. evolution debate remains unresolved among the general public (in the USA) because neither side has managed to speak the same language as the other side. My language is not formally defined in the sense of being a formal language, but it has formally defined ontological types.
Those are the labels used to describe the issue by the participants. But taking an outside view, the issue is inconsistent principles between the two sides. The fact that true religious believers reject the need for beliefs to pay rent in anticipated experience won't be solved by new vocabulary.