Yes, I agree that this can be a big problem. Right now, I expect to have made a change, but I can also imagine having made no change. I think there are actually observable properties that you can look at and use to predict how effective any non-trivial attempt to change someone's behavior with rational argument is going to be. Framing matters of course: if they aren't an intellectual, then you can't relate to them like one. But for example, high g and high metacognitive ability are indicators in my experience, and my friend has both, even if it's not on the level of many LWers, for example.
But an important thing to note is that I've sort of reinterpreted events to avoid flak from the commentariat. We're both blue-collar workers, so my everyday conversations don't look entirely like this; in fact, there's a lot more aggression involved. So, because my friend prides himself on his intelligence, and knows a little bit about rationality, and believes in the virtue of Hard Work, and I'm the only person in my social circle with a reputation for the ability to consistently generate deep responses and persuade anyone given enough time, I can say things like, "I'm going to keep breaking you down," and it's actually productive. And I really drive it home when he contradicts himself in non-obvious ways, especially when it's something like "You can be as rational as you want and all it will do is frustrate me more. I'm not trying to use my higher-order brain functions right now. I'm trying to vent not to get over it." And he's kind of amused by how irrational he knows he sounds, but keeps going anyway. And I'll say, "Bullshit. You laugh as you speak and you know how ridiculous you sound. You just sat there and talked about how you came up with reasons to get more angry. You're already using your higher-order brain functions. And I bet you're frustrated, but it's not the only thing that rationality is doing." And part of this is indicating that if he doesn't try harder to control his impulses, I won't just throw my hands up and mourn the fact that all was for naught; I'll get angry eventually, and he'll have to deal with that. In the end, the aggression does little harm, and lots of good. I find in lower socioeconomic classes that learning and teaching are more often of the form "Make X feel stupid for believing Y so that they believe Z instead, and do it in a lighthearted but slightly aggressive way." And there are better ways, but it works, and it's what many people know, so it's easier to use it than to change it all at once. I believe that this can be hard for some LWers to accept, because they're not used to being outside of high IQ, rationalist bubbles. In those bubbles, people are typically underconfident about their ability to self-regulate and underconfidence in that ability grants you status. But in my bubble, overconfidence in your ability to self-regulate is the norm, and overconfidence grants you status more often than underconfidence. So you have to act way more confident than you might in your usual epistemic mode, because this is the only way you can relate to someone who's extremely confident about their ability to self-regulate and who looks for confidence rather than a lack of it to evaluate the accuracy of someone's judgment in situations like these. And the overconfidence really isn't all that dangerous for a rationalist, because it's usually really simple stuff that can't result in astronomical failure. And likewise, you have to be willing to get angry, because you're playing chicken in a bubble where most people remove their steering wheels before they even put the key in the ignition.
It's also worth saying that I'm particularly sensitive to affect-laden situations like this because of people I lived with previously, so I also related that, and he is motivated not only by an understanding that he'll make me angry if he doesn't control his impulses enough, but that he'll cause me rather substantial emotional discomfort beyond that. Anyway, ultimately, I think stories of ineffectiveness are examples of people failing the Art, rather than the Art failing people. And obviously you have to pick worthwhile people to use your Art with. Some people really are extremely hard to reach or unreachable.
The article didn't reflect any of this because I was trying to make the content more relatable to my expected audience without offending their sensibilities or including a bunch of caveats that would make the article read in a contrived way.
(Content note: This is a story about one of the times that I've applied my understanding of rationality to reduce the severity of an affect-laden situation. This may remind you of Bayesian Judo, because it involves the mental availability and use of basic rationality techniques to perform feats that, although simple to perform in hindsight, leave an impression of surprising effectiveness on those who don't know what is generating the ability to perform the feats. However, I always felt dissatisfied with Bayesian Judo because it seemed dishonest and ultimately unproductive. Rationalists should exude not only auras of formidability, but of compassion. Read assured that the participants in this story leave mutually satisfied. I haven't read much about cognitive behavioral therapy or nonviolent communication, but this will probably look like that. Consider moving on to something else if what I've described doesn't seem like the sort of thing that would interest you.)
My friend lost his comb, and it was awful. He was in a frenzy for half an hour, searching the entire house, slamming drawers and doors as he went along. He made two phone calls to see if other people took his comb without asking. Every once in a while I would hear a curse or a drawn-out grunt of frustration. I kind-of couldn't believe it.
It makes more sense if you know him. He has a very big thing about people taking his possessions without asking, and the thing is insensitive to monetary value.
I just hid for a while, but eventually he knocked on my door and said that he 'needed to rant because that was the headspace he was in right now'. So he ranted about some non-comb stuff, and then eventually we got to the point where we mutually acknowledged that he was basically talking at me right now, and not with me, and that he was seriously pissed about that comb. So we started talking for real.
I said, "I can hardly imagine losing any one of my possessions and being as angry as you are right now. I mean, in particular, I never comb or brush my hair, so I can't imagine it in the most concrete possible sense, but even then, I can't imagine anything that I could lose that would make me react that way, except maybe my cellphone or my computer. The only way I can imagine reacting that way is if it was a consistent thing, and someone was consistently overstepping my boundaries by taking my things without asking, however cheap they were. I can't relate to this comb thing."
He said, "It's not about the comb, it's that I hate it when people take my stuff without asking. It really pisses me off. It would be different if I had just lost it, I wouldn't care. It's just like, "Why?" Why would you ever assume anything? Either you're right, and it's fine. Or you're wrong and you seriously messed up. Why would you ever not just ask?"
"Yeah, why?" I said. He didn't say anything.
I asked again, "Why?"
"What do you mean?"
"I mean if you were to really ask the question, non-rhetorically, "Why do people take things without asking?", what would the answer be?"
"Because they're just fundamentally inconsiderate. Maybe they were raised wrong or something."
I kind of smiled because I've tried to get him to notice black boxes in the past. He gets what I'm talking about when I bring it up, so I asked,
"Do you really think that that's what's going on in their heads? 'I'm going to be inconsiderate now.'? Do you really think there's a little 'evilness' node in their brains and that its value is jacked way up?"
"No, they probably don't even notice. They're not thinking they're gonna screw me over, they just never think about me at all. They're gathering things they need, and then they think 'Oh, I need a comb, better take it.' But it's my comb. That might be even worse than them being evil. I wouldn't have used the word 'inconsiderate' if I was talking about them being deliberate, I would have used a different word."
I replied, "Okay, that's an important distinction to make, because I thought of 'inconsiderateness' as purposeful. But I'm still confused, because when I imagine having my things taken because someone is evil, as opposed to having my things taken because someone made a mistake, I imagine being a lot more upset that my things were taken by evil than by chance. It's weird to me because you're experiencing the opposite. Why?"
He said, "It's not about why they took it, it's about the comb. Do you have any idea how much of an inconvenience that is? And if they had just thought about it, it wouldn't have happened. It just really pisses me off that people like that exist in the world. I specifically don't take other people's things. If someone takes your arm, through accident or evil, and they say "I took your arm because I'm a sadistic bastard who wanted to take your arm", or they just take your arm by being reckless and causing a car accident, then it doesn't matter. You'd still be like, "Yeah, and I don't have an arm right now. What do I do with that?""
I looked kind of amused, and said, "But I feel like the arm thing is a bad analogy, because it doesn't really fit the situation with the comb. Imagine if you could also misplace an arm, as you would any other object. That's...hard to imagine concretely. So, I'm still confused because you said before that you wouldn't have been as mad if you had just lost the comb. But now you're saying that you're mostly mad because of the inconvenience of not having the comb. So I don't really get it."
He thought for a minute and said, "Okay, yeah, that doesn't really make sense. I guess...maybe I was trying to look for reasons to get more pissed off about the whole thing and brought up the inconvenience of not having a comb? That was kind of stupid, I guess."
I said, "I really am curious. Please tell me, how much did the comb cost?"
"I got it for free with my shears!" He started laughing half-way through the sentence.
I laughed, and then I got serious again after a beat, and I continued, "And that's my main point. That something that costs so little and that wouldn't have riled you up if it wasn't so likely that it had been taken rather than misplaced, stresses both of us out on a Friday night, a time during which we've historically enjoyed ourselves. When the world randomly strikes at us and it's over before we can do anything, I feel like the only thing left to control is our reaction. It's not that people should never feel or express anger, or even that they shouldn't yell or slam things every once in a while, but that to keep it up for a long time or on a regular basis just seems like a cost with no benefit. And I don't want to sit in here suffering because I know one of my friends is suffering, unable to forget that all of this began with a missing comb, something that I would literally be willing to pay to replace. But that wouldn't have worked. And once again, this is not the same as someone stealing something extremely valuable or consistently violating your personal boundaries."
He sighed. And then he said somberly,
"I just wish...that I lived in a world where my cup runneth over with comb." And we both laughed. And the tension was gone.