The basic questions of rationality

I've been on Less Wrong since its inception, around March 2009. I've read a lot and contributed a lot, and so now I'm more familiar with our jargon, I know of a few more scientific studies, and I might know a couple of useful tricks. Despite all my reading, however, I feel like I'm a far cry from learning rationality. I'm still a wannabe, not an amateur. Less Wrong has tons of information, but I feel like I haven't yet learned the answers to the basic questions of rationality.

I, personally, am a fan of the top-down approach to learning things. Whereas Less Wrong contains tons of useful facts that could, potentially, be put together to answer life's important questions, I really would find it easier if we started with the important questions, and then broke those down into smaller pieces that can be answered more easily.

And so, that's precisely what I'm going to do. Here are, as far as I can tell, the basic questions of rationality—the questions we're actually trying to answer here—along with what answers I've found:

Q: Given a question, how should we go about answering it? A: By gathering evidence effectively, and correctly applying reason and intuition.

Q: How can we effectively gather relevant evidence? A: I don't know. (Controlled experiments? Asking people?)
Q: How can we correctly apply reason? A: If you have infinite computational resources available, use probability theory.
- Q: We don't have infinite computational resources available, so what now? A: I don't know. (Apply Bayes' rule anyway? Just try to emulate what a hypercomputer would do?)
Q: How can we successfully apply intuition? A: By repairing our biases, and developing habits that point us in the right direction under specific circumstances.
- Q: How can we find our biases? A: I don't know. (Read Less Wrong? What about our personal quirks? How can we notice those?)
- Q: Once we find a bias, how can we fix it? A: I don't know. (Apply a correction, test, repeat? Figure out how the bias feels?)
- Q: How can we find out what habits would be useful to develop? A: I don't know. (Examine our past successes and rationalize them?)
- Q: Once we decide on a habit, how can we develop it? A: I don't know. (Sheer practice?)

We could answer some of these questions ourselves, though simple practice and straightforward methods. The method "apply a correction, test, repeat", for example, is so generally useful that it deserves to be called the Fundamental Algorithm of Control. Nevertheless, since Less Wrong is devoted to developing human rationality, surely it contains answers to these questions somewhere. Where are they?

Q: Given a question, how should we go about answering it? A: By gathering evidence effectively, and correctly applying reason and intuition.

Q: How can we effectively gather relevant evidence? A: I don't know. (Controlled experiments? Asking people?)
Q: How can we correctly apply reason? A: If you have infinite computational resources available, use probability theory.
- Q: We don't have infinite computational resources available, so what now? A: I don't know. (Apply Bayes' rule anyway? Just try to emulate what a hypercomputer would do?)
Q: How can we successfully apply intuition? A: By repairing our biases, and developing habits that point us in the right direction under specific circumstances.
- Q: How can we find our biases? A: I don't know. (Read Less Wrong? What about our personal quirks? How can we notice those?)
- Q: Once we find a bias, how can we fix it? A: I don't know. (Apply a correction, test, repeat? Figure out how the bias feels?)
- Q: How can we find out what habits would be useful to develop? A: I don't know. (Examine our past successes and rationalize them?)
- Q: Once we decide on a habit, how can we develop it? A: I don't know. (Sheer practice?)

Q: How do you notice when there is a question to ask?

An answer to my own question: by learning to notice the tiny sensation that something is not quite right, and attending to it as urgently as you would search out the source of an unexpected smell of burning.

Rationality has many tools, but they are all grasped with the handle of attention.

40

The basic questions of rationality

40

40

40

The basic questions of rationality

40

40