Can anyone think of good ways to notice when outright deception is being used? How could a rationalist practice her skills at a magic show?
Most "rationalists" are quite smart people, so tricks that are designed by a trickster to fool the masses rarely work on us. For example, I doubt that many on this site would invest heavily in a pyramid scheme or get fooled by a used car salesman. This is because these tricks are targeted at the average idiot.
However, I have recently noticed that there is, for each of us, a stalker who stalks us and at each and every turn attempts to deceive us, and is just as smart as we are. That stalker/trickster is your own cognitive biases, and by far and away inflicts the greatest material losses on you. This is certainly true in my case.
I cannot even remember the last time I was fooled by someone else, but now that I am working on reducing my losses due to self deception, I realize that basically every day I engage in successful self-deception: I get into some emotional state, myopic, irrational algorithms take over, and I make up little excuses to myself for why they reached the right conclusion.
The real enemy is already inside your head.
Most "rationalists" are quite smart people, so tricks that are designed by a trickster to fool the masses rarely work on us.
Wrong. Tricksters rely on people making stupid assumptions and failing to check assertions. People with a lot of brainpower can do those things just as easily as people without.
Physicists asked to evaluate paranormal claims do very poorly, yet they are clearly very brainy. It takes more than just brains to be intelligent - you have to use the brains properly.
If I had a dollar for every brainy person who'd been gulled because they thought they were "too smart" to require being skeptical...
Related: Trust in Math
I was reading John Allen Paulos' A Mathematician Plays the Stock Market, in which Paulos relates a version of the well-known "missing dollar" riddle. I had heard it once before, but only vaguely remembered it. If you don't remember it, here it is:
I remembered that the solution involves trickery, but it still took me a minute or two to figure out where it is. At first, I started mentally keeping track of the dollars in the riddle, trying to see where one got dropped so their sum would be 30.
Then I figured it out. The story should end:
I told my fiance the riddle, and asked her where the missing dollar went. She went through the same process as I did, looking for a place in the story where $1 could go missing.
It's remarkable to me how blatantly deceptive the riddle is. The riddler states or implies at the end of the story that the dollars paid by the guests and the dollars kept by the bellhop should be summed, and that that sum should be $30. In fact, there's no reason to sum the dollars paid by the guests and the dollars kept by the bellhop, and no reason for any accounting we do to end up with $30.
The contrasts somewhat with the various proofs that 1 = 2, in which the misstep is hidden somewhere within a chain of reasoning, not boldly announced at the end of the narrative.
Both Paulos and Wikipedia give examples with different numbers that make the deception in the missing dollar riddle more obvious (and less effective). In the case of the missing dollar riddle, the fact that $25, $27, and $30 are close to each other makes following the incorrect path very seductive.
This riddle made me remember reading about how beginning magicians are very nervous in their first public performances, since some of their tricks involve misdirecting the audience by openly lying (e.g., casually pick up a stack of cards shuffled by a volunteer, say "Hmm, good shuffle" while adding a known card to the top of the stack, hand the deck back to the volunteer, and then boldly announce "notice that I have not touched or manipulated the cards!"1). However, they learn to be more comfortable once they find out how easily the audience will pretty much accept whatever false statements they make.
Thinking about these things makes me wonder about how to think rationally given the tendency for human minds to accept some deceptive statements at face value. Can anyone think of good ways to notice when outright deception is being used? How could a rationalist practice her skills at a magic show?
How about other examples of flagrant misdirection? I suspect that political debates might be able to make use of such techniques (I think that there might be some examples in the recent debates over health care reform accounting and the costs of obesity to the health care system, but I haven't been able to find any yet.)
Footnote 1: I remember reading this example very recently, maybe at this site. Please let me know whom to credit for it.