The intuitive breakthrough for me was realizing that given a proposition P and an argument A that supports or opposes P, then showing that A is invalid has no effect on the truth or falsehood of P, and showing that P is true has no effect on the validity of A. This is the core of the "knowing biases can hurt you" problem, and while it's obvious if put in formal terms, it's counterintuitive in practice. The best way to get that to sink in, I think, is to practice demolishing bad arguments that support a conclusion you agree with.
The intuitive breakthrough for me was realizing that given a proposition P and an argument A that supports or opposes P, then showing that A is invalid has no effect on the truth or falsehood of P
That sort of makes sense if what you mean is "whatever we humans think about A has no effect on the truth or falsehood of P in a Platonic sense" but surely showing that A is invalid ought to change how likely you think that P is true?
and showing that P is true has no effect on the validity of A.
Similarly, if P is actually true, a random argument t...
My current plan does still call for me to write a rationality book - at some point, and despite all delays - which means I have to decide what goes in the book, and what doesn't. Obviously the vast majority of my OB content can't go into the book, because there's so much of it.
So let me ask - what was the one thing you learned from my posts on Overcoming Bias, that stands out as most important in your mind? If you like, you can also list your numbers 2 and 3, but it will be understood that any upvotes on the comment are just agreeing with the #1, not the others. If it was striking enough that you remember the exact post where you "got it", include that information. If you think the most important thing is for me to rewrite a post from Robin Hanson or another contributor, go ahead and say so. To avoid recency effects, you might want to take a quick glance at this list of all my OB posts before naming anything from just the last month - on the other hand, if you can't remember it even after a year, then it's probably not the most important thing.
Please also distinguish this question from "What was the most frequently useful thing you learned, and how did you use it?" and "What one thing has to go into the book that would (actually) make you buy a copy of that book for someone else you know?" I'll ask those on Saturday and Sunday.
PS: Do please think of your answer before you read the others' comments, of course.