This reminds me of the HPMOR chapter in which Harry tests Hermione's hypothesis checking skills. What you're telling me is just what you'd expect if people were easily convinced of pretty much anything (with some caveats, admittedly), given social capital and social investment (which you have mentioned in your initial explanation). You have a specialized mechanism-of-action to explain your apparent success, one which indeed may not be easily adaptable to other ventures.
The problem is that it doesn't explain the ubiquitous occurrence of people being convinced of pretty much any topic you could imagine (requiring more specialized theories). From organized religion, cults, homeopathy, nationalism, anti-nationalism, consumerism, anti-consumerism, big weddings, small weddings, no weddings, monogamy, polygamy, psychic energy, Keynesianism, Austrian economics, rationality, anti-rationality, the list goes on. It doesn't matter that some of these happen to be correct when their exact opposite is also on the list, with plenty of adherents.
You have the anecdote, but looking at the human condition, I see plenty of data to the opposite. Though if you interpret that differently, please share.
I'm not satisfied if they agree today and disagree tomorrow.
Do you think that when (some time after your rationality talk with them) they display a bias in a real life situation, and you kindly make them notice that they did, that they'll agree and have learned a lesson? It's all good as long as it's in the abstract, and a friendly guy who is sharing a cool topic he's passionate about.
Which is good in a way, because just as the first rationality talk doesn't stick, neither does the first e.g. "convert to your girlfriend's religion", usually.
Also, what, in your opinion, are the relative weights you'd ascribe to your success, in terms of "social investment / social strategy" versus your System 2/System 1 approach?
I would be interested in you actually trying the real, falsifying experiment: convincing someone of something obviously false (to you). It's not hard, in the general case. Though, as you say, in recent years it has become slightly harder in some ways, easier in others: Far from creating one shared space, today's interconnectivity seems to have led to a bunch of echo-chamber bubbles, even if Wikipedia is a hopeful sign.
Then again, Wikipedia exists. As does the multi-billion (and growing) homeopathy market.
The problem is that it doesn't explain the ubiquitous occurrence of people being convinced of pretty much any topic you could imagine (requiring more specialized theories).
It's quite easy to say in the abstract that people can be persuaded. It's quite different to see what effort it takes to convince another person in real life.
I think we all have conversation where we try to convince someone and fail.
It seems that the CFAR workshops so far have been dedicated to people who have preconceptions pretty close in ideaspace to the sorts of ideas proposed on LW and by the institutions related to it. This is not a criticism; it's easier to start out this way: as has been said, in a different context and perhaps not in so many words, we should focus on precision before tractability. We're not going to learn a thing about the effectiveness of rationality training from people who won't even listen to what we have to say. Nevertheless, there will come a day when these efforts must be expanded to people who don't already view us as high in social status, so we still have to solve the problem of people being more concerned with both our and their social status than with listening to what we have to say. I propose that the solution is to divorce the consideration of social status from the argument.
There is a lot of talk of cognitive biases on LW, and for good reason, but ultimately what we are trying to teach people is that they are prone to misinterpreting reality, and cognitive biases are only one component of this. One of the problems with trying to teach people about biases is that people feel personally responsible for being biased; many people have a conception of thinking as an 'active' process, so they feel as though it reflects upon their character. On the other hand, many people conceive of perception as a 'passive' process; no one feels personally responsible for what they perceive. So, I propose that we circumvent this fear of character assassination by demonstrating how people can misinterpret reality through perception. Enter: the rubber hand illusion.
In case you're unfamiliar with this illusion, to demonstrate the rubber hand illusion, a subject sits at a table, a rubber hand is placed in front of them, oriented relative to their body as a natural hand would be, and a partition is placed between the rubber hand and their 'real' hand such that they are unable to see the 'real' hand. Then, the experimenter simultaneously 'stimulates' both hands at random intervals (usually by stroking each hand with a paintbrush). Then, the experimenter overextends the tips of a finger on each hand, the rubber hand about 90 degrees, and the 'real' hand about 20 degrees (it's not really overextension, and it wouldn't cause pain outside of the experiment's conditions). Measurements of skin conductance response indicate that subjects anticipate pain when this is done, and a very small selection of subjects even report actually experiencing pain. Also, (just for kicks) when subjects are questioned about the degree to which they believe their 'real' finger was bent, they overestimate, by an average of about 20 degrees.
As Dr. Vilayanur Ramachandran has demonstrated, the rubber hand illusion isn't the most general example of this sort of illusion: the human mind can even anticipate pain from injury to the surface of a table. In fact, there is evidence that the human mind's evaluation of what is and is not part of its body isn't even dependent upon distance: Dr. Ramachandran has also demonstrated this with rubber hands attached to unnaturally long rubber arms.
I think that there are also three beneficial side effects to this exercise. (1) We are trying to convince people that Bayesian inference is a useful way to form beliefs, and this illusion demonstrates that every human mind already unconsciously uses Bayesian inference all of the time (namely, to infer what is and isn't its body). To further demonstrate the part about Bayesian inference, I would suggest that subjects also subsequently be shown how the illusion does not occur when the rubber hand is perpendicular to the 'real' hand or when the 'stimulations' aren't simultaneous. (2) After the fact, the demonstration grants social status to the demonstrator in the eyes of the subject: "This person showed me something that I consider extremely significant and that I didn't know about, therefore, they must be important." (3) Inconsistencies in perception instill feelings of self-doubt and incredulity, which makes it easier to change one's mind.
Addendum: This post has been substantially edited, both for brevity and on the basis of mistakes mentioned in the comments, such that some of the comments now appear nonsensical. Here is a draft that I found on my desktop which as far as I can tell is identical to the original post: http://pastebin.com/BL81VQVp