The problem is that most opinions people hold, even those of LessWrong's users, are already based on filtered evidence. If confirmation bias wasn't the default state of human affairs, it wouldn't be a problem so noteworthy as to gain widespread understanding. (There are processes that can cause illegitimate spreading, but that isn't the case with confirmation bias.) When you sit down to do the exercise and realize legitimate arguments (not merely ad hoc arguments) against your own views, you're overcoming your confirmation bias (default) on that issue for the first time. This is why it is important to respect your partner in debate; without respecting their ability to reason and think things you haven't, their mere disagreement with your permanent correctness directly causes condescension. Nonsensical ad-hoc arguments are more useful than no argument whatsoever; one has the quality of provoking thought. The only way otherwise rational people come to disagree is from the differing priors of their respective data sets; it's not that the wrong one among them is thinking up nonsense and being negatively affected by it.
The truth is I don't really read comments on LessWrong all that much. I can't stand it. All I see being discussed and disagreed over are domain-specific trivial arguments. I recall someone on IRC once criticized that they hadn't seen evidence that Eliezer_Yudkowsky ever really admits being wrong in the face of superior arguments. This same concept applies to the entirety of LessWrongers; nobody is really changing their deep beliefs after "seeing the light." They're seeing superior logic and tactics and adding those onto their model. The model still remains the same, for the most part. Politics is only a mind-killer insofar as the participants in the discussion are unable to correct their beliefs on physically and presently important issues. That there exist subjects that LessWrong's users ban themselves from participation in is class A evidence of this. LessWrong only practices humble rationality in the realm of things that are theoretically relevant. The things that are actually shown to matter are taboo to even bring up because that might cause people to "realize" (confirmation bias) that they're dealing with people they consider to be idiots. Slow progress is being made in terms of rationality by this community, but it is so cripplingly slow by my standards that it frustrates me. "You could do so much better if you would just accept this one single premise as plausible!" The end result is that LessWrong is advancing, yes, but not at a pace that exceeds the bare minimum of the average.
Everything this community has done up to now is a good warm-up, but now I'd like to start seeing some actual improvement where it counts.
It's not the mere existence of inferential silence that is the issue here. Inferential silence exists everywhere on digital forums. What's relevant is the exact degree to which the inferential silence occurs. For example, if nobody commented, upvoted, or downvoted, then LessWrong is just a disorganized blog. If all the topics worth discussing have their own posts and nobody posts anything new, and everyone stops checking for new content, the site is effectively dead. The measuring of inferential silence has the same purpose as asking, "Is this site useful to me?" Banned subjects are a severe form of inferential silence. We're rationalists. We ought to be able to discuss any subject in a reasonable manner. Other places, when someone doesn't care about a thread, they just don't bother reading it. Here, you're told not to post it. Because it's immoral to distract all these rationalists who are supposed to be advancing the Singularity with temptation to debate (seemingly) irrelevant things. LessWrong places next to no value on self-restraint; better to restrain the world instead.
This is the part where things get difficult to navigate.
I predicted your reaction of considering the coherency of the collective as overblown. I'd already started modeling responses in my head when I got up from the computer after posting the comment. I don't predict you're terribly bothered by a significant degree of accuracy to the prediction; rather, I predict that, to you, it will seem only obvious that I should have been able to predict that. This will all seem fairly elementary to you. What I'm unsure about is the degree to which you are aware that you stand out from the rest of these folks. You're exhibiting a deeper level of understanding of the usefulness of epistemic humility in bothering to speak to me and read my comments in the way that you are. You offer conversational pleasantries and pleasant offers for conversation, but that can be both a consciously recognized utility or an unconscious one, with varying degrees in between. I can already tell, though, what kind of path you've been on with that behavior. You'll have seen and experienced things that most other LWers have not. Basically, what sets you apart is that you don't suck at conversation.
It's not that I predicted that you'd disagree or be unsure about what I was referring to, it's more that the idea I understand, by virtue of being able to understand it, inherently lets me know that you will immediately agree if you can actually grasp the concept. It's not that you'll immediately see everything I have; that part will take time. What will happen is that you'll have grasped the concept and the means to test the idea, though you'll feel uncertain about it. You'll of course be assessing the data you collect in the opposite of the manner that I do; while I search for all the clues indicating negatives, you'll search for clues and reasoning that leave LessWrong with less blame—or maybe you'll try to be more neutral about it (if you can determine where the middle ground lies). I wrote my last comment because I'd already concluded that you'll be able to fully grasp my concept; but be forewarned: Understanding my lone hypothesis in light of no competing hypotheses could change your beliefs! (An irreparable change, clearly.)
I've more to say, but it won't make sense to say it without receiving feedback about the more exact mechanics of your stage of grasping my concept. I predict you won't notice anything out of the ordinary about the thoughts you'll have thought in reading/responding to/pondering this. These prediction, again, will appear to be mundane.
When you sit down to do the exercise and realize legitimate arguments (not merely ad hoc arguments) against your own views, you're overcoming your confirmation bias (default) on that issue for the first time.
That's not obvious to me. I'd expect LWers to be the kind of high-NFC/TIE people who try to weigh evidence in a two-sided way before deciding to like a particular politician or organization in the first place, and would probably, having made that decision, try to remain aware opposing evidence exists.
...Nonsensical ad-hoc arguments are more useful th
Making fun of things is actually really easy if you try even a little bit. Nearly anything can be made fun of, and in practice nearly anything is made fun of. This is concerning for several reasons.
First, if you are trying to do something, whether or not people are making fun of it is not necessarily a good signal as to whether or not it's actually good. A lot of good things get made fun of. A lot of bad things get made fun of. Thus, whether or not something gets made fun of is not necessarily a good indicator of whether or not it's actually good.[1] Optimally, only bad things would get made fun of, making it easy to determine what is good and bad - but this doesn't appear to be the case.
Second, if you want to make something sound bad, it's really easy. If you don't believe this, just take a politician or organization that you like and search for some criticism of it. It should generally be trivial to find people that are making fun of it for reasons that would sound compelling to a casual observer - even if those reasons aren't actually good. But a casual observer doesn't know that and thus can easily be fooled.[2]
Further, the fact that it's easy to make fun of things makes it so that a clever person can find themselves unnecessarily contemptuous of anything and everything. This sort of premature cynicism tends to be a failure mode I've noticed in many otherwise very intelligent people. Finding faults with things is pretty trivial, but you can quickly go from "it's easy to find faults with everything" to "everything is bad." This tends to be an undesirable mode of thinking - even if true, it's not particularly helpful.
[1] Whether or not something gets made fun of by the right people is a better indicator. That said, if you know who the right people are you usually have access to much more reliable methods.
[2] If you're still not convinced, take a politician or organization that you do like and really truly try to write an argument against that politician or organization. Note that this might actually change your opinion, so be warned.