I suppose it was only a matter of time before less wrong found the "fnords". Although at moment we seem to be obsessed with the silly or superficial ones. There is an art, parallel to the core art of rationality, in learning to see the assumptions and deceptions we build up in order to function, the accretion of simplifications and half-answers which become unchallenged beliefs so basic that we forget we even believe them.
And as the saying goes once you see the fnords, you see them everywhere.
And there are so many to see, under a thin coat of fear or denial, an idea or a correction lies ready to be revealed. These are cheap, although filled with the thrill of danger, because they frighten and inspire in equal measure.
But deep underneath all that, are the big ones, looming, twisting things which have tunneled their way through our knowledge and practices. These are the ones you need to ignore because they don't just frighten or require accepting what others deny, they mean functionally shifting your entire reference frame. It is as though language itself conspires to make these deeply embedded assumptions and delusions into something inexpressible, weird at best, madness at worse.
If you search patiently and carefully enough you can start to find them. You can even catalog or map them, seeing how they unfold into each other. But that doesn't mean you've figured out how the express them. How do you show them in a way that provides nearly as much engagement as a stream of rationalizations? That's something I'm still working on.
What's an example? [How about one small one and one big one?]
This article is a deliberate meta-troll. To be successful I need your trolling cooperation. Now hear me out.
In The Strangest Thing An AI Could Tell You Eliezer talks about asognostics, who have one of their arm paralyzed, and what's most interesting are in absolute denial of this - in spite of overwhelming evidence that their arm is paralyzed they will just come with new and new rationalizations proving it's not.
Doesn't it sound like someone else we know? Yes, religious people! In spite of heaps of empirical evidence against existence of their particular flavour of the supernatural, internal inconsistency of their beliefs, and perfectly plausible alternative explanations being well known, something between 90% and 98% of humans believe in the supernatural world, and is in a state of absolute denial not too dissimilar to one of asognostics. Perhaps as many as billions of people in history have even been willing to die for their absurd beliefs.
We are mostly atheists here - we happen not to share this particular delusion. But please consider an outside view for a moment - how likely is it that unlike almost everyone else we don't have any other such delusions, for which we're in absolute denial of truth in spite of mounting heaps of evidence?
If the delusion is of the kind that all of us share it, we won't be able to find it without building an AI. We might have some of those - it's not too unlikely as we're a small and self-selected group.
What I want you to do is try to trigger absolute denial macro in your fellow rationalists! Is there anything that you consider proven beyond any possibility of doubt by both empirical evidence and pure logic, and yet saying it triggers automatic stream of rationalizations in other people? Yes, I pretty much ask you to troll, but it's a good kind of trolling, and I cannot think of any other way to find our delusions.