Q: Given a question, how should we go about answering it? A: By gathering evidence effectively, and correctly applying reason and intuition.
An important point omitted in the proposed answer: Reduce the question into subquestions stated in primitively testable terms. Try to dispel confusions. Make sure the question even makes sense. For a large class of questions (all classical philosophical and ethical questions, most political ones) starting to gather evidence before the question is reduced is a mistake. This is perhaps the most important idea which can be learned from LW.
Q: How can we effectively gather relevant evidence?
When it is clear what relevant evidence is (whatever fact whose probability strongly depends on the tested hypothesis) and when a reliable intuitive understanding of probability is at hand, this should be easy. (Of course, depending on what level of effectivity are you aiming at.) Most common errors in reasoning don't stem from lack of evidence, but from incorrect intuitive probabilistic analysis. Creationists often know a lot of relevant facts.
Q: We don't have infinite computational resources available, so what now? A: I don't know. (Apply Bayes' rule anyway? Just try to emulate what a hypercomputer would do?)
Apply Bayes' rule anyway. The result will not be perfect and you should be aware of that, but in majority of situations it's still an improvement over intuitive guesses.
Q: How can we find our biases? A: I don't know. (Read Less Wrong? What about our personal quirks? How can we notice those?)
It's hard to do perfectly, of course. But simply by learning about several standard biases I was able to spot such patterns of reasoning in my thoughts and I believe I don't commit them now so often as in the past. Personal quirks? Listen to feedback you get from others.
Q: Once we find a bias, how can we fix it?
Retreat to more formalised reasoning, if possible.
Apply Bayes' rule anyway. The result will not be perfect and you should be aware of that, but in majority of situations it's still an improvement over intuitive guesses.
How do you determine the relevant probabilities? What if you're looking for, say, the probability of a nuclear attack occurring anywhere in the world in the next 20 years?
Q: Once we find a bias, how can we fix it?
Retreat to more formalised reasoning, if possible.
Yes, but that doesn't remove the bias. Surely if it's at all possible to remove a bias, that's better than circumventing...
I've been on Less Wrong since its inception, around March 2009. I've read a lot and contributed a lot, and so now I'm more familiar with our jargon, I know of a few more scientific studies, and I might know a couple of useful tricks. Despite all my reading, however, I feel like I'm a far cry from learning rationality. I'm still a wannabe, not an amateur. Less Wrong has tons of information, but I feel like I haven't yet learned the answers to the basic questions of rationality.
I, personally, am a fan of the top-down approach to learning things. Whereas Less Wrong contains tons of useful facts that could, potentially, be put together to answer life's important questions, I really would find it easier if we started with the important questions, and then broke those down into smaller pieces that can be answered more easily.
And so, that's precisely what I'm going to do. Here are, as far as I can tell, the basic questions of rationality—the questions we're actually trying to answer here—along with what answers I've found:
Q: Given a question, how should we go about answering it? A: By gathering evidence effectively, and correctly applying reason and intuition.