Well, the mind dump from earlier was mostly food for thought, not a staking out claims or definitions. I guess my rough definition of intelligence fits what I find in the dictionary:
The ability to acquire and apply knowledge and skills
The same dictionary, however, defines rationality as a form of the word rational:
Based on or in accordance with reason or logic
I take intelligence to mean, "the ability to accomplish stuff," and rationality to mean, "how to get intelligence." Abstracted, rationality more or less becomes, "how to get the ability to accomplish stuff." This is contrasted with "learning" which is:
Gaining or acquiring knowledge of or skill in (something) by study, experience, or being taught
I am not proposing this definition of rationality is what anyone else should use. Rather, it is a placeholder concept until I feel comfortable sitting down and tackling the problem as a whole. Right now I am still in aggregation mode which is essentially collecting other people's thoughts on the subject.
Honestly, all of this discussion is interesting but it may not be helpful. I think Eliezer's concept of the nameless virtue is good to keep in mind during these kinds of discussions:
You may try to name the highest principle with names such as “the map that reflects the territory” or “experience of success and failure” or “Bayesian decision theory”. But perhaps you describe incorrectly the nameless virtue. How will you discover your mistake? Not by comparing your description to itself, but by comparing it to that which you did not name.
A currently existing social norm basically says that everyone has the right to an opinion on anything, no matter how little they happen to know about the subject.
But what if we had a social norm saying that by default, people do not have the right to an opinion on anything? To earn such a right, they ought to have familiarized themselves on the topic. The familiarization wouldn't necessarily have to be anything very deep, but on the topic of e.g. controversial political issues, they'd have to have read at least a few books' worth of material discussing the question (preferrably material from both sides of the political fence). In scientific questions where one needed more advanced knowledge, you ought to at least have studied the field somewhat. Extensive personal experience on a subject would also be a way to become qualified, even if you hadn't studied the issue academically.
The purpose of this would be to enforce epistemic hygiene. Conversations on things such as public policy are frequently overwhelmed by loud declarations of opinion from people who, quite honestly, don't know anything on the subject they have a strong opinion on. If we had in place a social norm demanding an adequate amount of background knowledge on the topic before anyone voiced an opinion they expected to be taken seriously, the signal/noise ratio might be somewhat improved. This kind of a social norm does seem to already be somewhat in place in many scientific communities, but it'd do good to spread it to the general public.
At the same time, there are several caveats. As I am myself a strong advocate on freedom of speech, I find it important to note that this must remain a *social* norm, not a government-advocated one or anything that is in any way codified into law. Also, the standards must not be set *too* high - even amateurs should be able to engage in the conversation, provided that they know at least the basics. Likewise, one must be careful that the principle isn't abused, with "you don't have a right to have an opinion on this" being a generic argument used to dismiss any opposing claims.