The intuitive breakthrough for me was realizing that given a proposition P and an argument A that supports or opposes P, then showing that A is invalid has no effect on the truth or falsehood of P
That sort of makes sense if what you mean is "whatever we humans think about A has no effect on the truth or falsehood of P in a Platonic sense" but surely showing that A is invalid ought to change how likely you think that P is true?
and showing that P is true has no effect on the validity of A.
Similarly, if P is actually true, a random argument that concludes with "P is true" is more likely to be valid than a random argument that concludes with "P is false". So showing P is true ought to make you think that A is more or less likely to be valid depending on its conclusion.
(Given that this comment was voted up to 3 and nobody gave a counterargument, I wonder if I'm missing something obvious.)
3 is still a small number. If it were 10+ then you should worry. I'm confused by this too.
The nearest correct idea I can think of to what Jim actually said, is that if you have a proposition P with an associated credence based on the available evidence, then finding an additional but invalid argument A shouldn't affect your credence in P. The related error is assuming that if you argue with someone and are able to demolish all their arguments, that this means that you are correct, and giving too little weight to the possibility that they are a bad arguer w...
My current plan does still call for me to write a rationality book - at some point, and despite all delays - which means I have to decide what goes in the book, and what doesn't. Obviously the vast majority of my OB content can't go into the book, because there's so much of it.
So let me ask - what was the one thing you learned from my posts on Overcoming Bias, that stands out as most important in your mind? If you like, you can also list your numbers 2 and 3, but it will be understood that any upvotes on the comment are just agreeing with the #1, not the others. If it was striking enough that you remember the exact post where you "got it", include that information. If you think the most important thing is for me to rewrite a post from Robin Hanson or another contributor, go ahead and say so. To avoid recency effects, you might want to take a quick glance at this list of all my OB posts before naming anything from just the last month - on the other hand, if you can't remember it even after a year, then it's probably not the most important thing.
Please also distinguish this question from "What was the most frequently useful thing you learned, and how did you use it?" and "What one thing has to go into the book that would (actually) make you buy a copy of that book for someone else you know?" I'll ask those on Saturday and Sunday.
PS: Do please think of your answer before you read the others' comments, of course.