Good post.
How can we effectively gather relevant evidence?
Break it down:
What is evidence? How do we gather that?
Just try to emulate what a hypercomputer would do?
No. RYK is a terrible chess program, even worse than Yudkowsky.
Every so often, answer a question twice, once using formalisms in some place, the other times using intuition in that place. If they output different answers, notice your confusion. Do not be confident in any course of action (I do not say do not take action) until you think you know why they gave different answers. Learn about formalisms and intuition until you know where you went wrong - and if you got different answers you certainly went wrong somewhere, if you got the same answers you possibly went wrong somewhere.
Apply formalisms and intuition separately.
This seems wrong, in exactly the same way that "apply reasoning and observation separately" is wrong.
I've been on Less Wrong since its inception, around March 2009. I've read a lot and contributed a lot, and so now I'm more familiar with our jargon, I know of a few more scientific studies, and I might know a couple of useful tricks. Despite all my reading, however, I feel like I'm a far cry from learning rationality. I'm still a wannabe, not an amateur. Less Wrong has tons of information, but I feel like I haven't yet learned the answers to the basic questions of rationality.
I, personally, am a fan of the top-down approach to learning things. Whereas Less Wrong contains tons of useful facts that could, potentially, be put together to answer life's important questions, I really would find it easier if we started with the important questions, and then broke those down into smaller pieces that can be answered more easily.
And so, that's precisely what I'm going to do. Here are, as far as I can tell, the basic questions of rationality—the questions we're actually trying to answer here—along with what answers I've found:
Q: Given a question, how should we go about answering it? A: By gathering evidence effectively, and correctly applying reason and intuition.