First of all, let me say that rationalist-to-rationalist conversations don't really have this problem. This is all about what happens when we talk to people who think less in "far-mode" if you will. What I've found is when talking with non-rationalists, I have to consciously switch into a different mindset to get "flow". Let me give a personal example.
I was once at a bar where I met some random people, and one of them told me something that a rationalist would consider "woo". He explained that he'd read that an atomic bomb was a particularly terrible thing, because unlike when you die normally, the radiation destroys the souls. I paused for a moment, swallowed all my rationalist impulses, and thought: "Is there anyway what he said could be meaningful?" I responded: "Well the terrible thing about an atomic explosion, is that it kills not just a person in isolation, but whole families, whole communities... if just one person dies, their friends and families can respect that person's death and celebrate their memories, but when that many people die all at once, their entire history of who they are, their souls, are just erased in an instant". He told me that was deep, and bought me a drink.
Did I feel dishonest? Not really. I decided what was relevant and what was not. Obviously the bit he said about radiation didn't make scientific sense, but I didn't feel the reason he'd brought up the idea was that he cared for a science lesson. Similarly, I could have asked the question: "well exactly what do you mean by a 'soul'?" Instead I chose an interpretation that seemed agreeable to both of us. Now, had he specifically asked me for an analytical opinion, I would have absolutely given him that. But for now, what I'd done had earned myself some credibility, so that later in the conversation, if I wanted to be persuasive of an important "rationalist" opinion, I'd actually be someone worth listening too.
Yes of course you should not habitually divert conversations into you lecturing others on how some specific thing they said might be in error. For example, when I listen to an academic talk I do not speak up about most of the questionable claims made - I wait until I can see the main point of the talk and see what questionable claims might actually matter for their main claim. Those are the points I consider raising. Always keep in mind the purpose of your conversation.
The product of Less Wrong is truth. However, there seems to be a reluctance of the personality types here - myself included - to sell that product. Here's my evidence:
We actually label many highly effective persuasive strategies that can be used to market our true ideas as "dark arts". What's the justification for this negative branding? A necessary evil is not evil. Even if - and this is a huge if - our future utopia is free of dark arts, that's not the world we live in today. Choosing not to use them is analogous to a peacenik wanting to rid the world of violence by suggesting that police not use weapons.
We treat our dislike of dark arts as if it's a simple corollary of the axiom of the virtue of truth. Does this mean we assume the ends (more people believe the truth) doesn't justify the means (persuasion to the truth via exploiting cognitive biases)? Or are we just worried about being hypocrites? Whatever the reason, such an impactful assumption deserves an explanation. Speaking practically, the successful practice of dark arts requires the psychological skill of switching hats, to use Edward de Bono's terminology. While posting on Less Wrong, we can avoid and are in fact praised for avoiding dark arts, but we need to switch up in other environments, and that's difficult. Frankly, we're not great at it, and it's very tempting to externalize the problem and say "the art is bad" rather than "we're bad at the art".
Our distaste for rhetorical tactics, both aesthetically and morally, profoundly affects the way we communicate. That distaste is tightly coupled with the mental habit of always interpreting the value of what is said purely for its informational content, logical consistency, and insight. I'm basing the following question on my own introspection, but I wonder if this almost religiously entrenched mental habit could make us blind to the value of the art of persuasion? Let's imagine for a moment, the most convincing paragraph ever written. It was truly a world-wonder of persuasion - it converted fundamentalist Christians into atheists, suicide bombers into diplomats, and Ann Coulter-4-President supporters into Less Wrong sycophants. What would your reaction to the paragraph be? Would you "up-vote" this work of genius? No way. We'd be competing to tell the fundamentalist Christian that there were at least three argument fallacies in the first sentence, we'd explain to the suicide bomber that the rhetoric could be used equally well to justify blowing us all up right now, and for completeness we'd give the Ann Coulter supporter a brief overview of Bayesianism.