David_Gerard comments on How To Lose 100 Karma In 6 Hours -- What Just Happened - Less Wrong

-31 Post author: waitingforgodel 10 December 2010 08:27AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (214)

You are viewing a single comment's thread. Show more comments above.

Comment author: David_Gerard 10 December 2010 02:11:30PM *  14 points [-]

Not really :-) If you keep awareness of the cult attractor and can think of how thinking these things about an idea might trip you up, that's not a flawless defence but will help your defences against the dark arts.

What inspired you to the phrase "invincible mind fortresses"? I like it. Everyone thinks they live in one, that they're far too intelligent/knowledgeable/rational/Bayesian/aware of their biases/expert on cults/etc to fall into cultishness. They are of course wrong, but try telling them that. (It's like being smart enough to be quite aware of at least some of your own blithering stupidities.)

(I read the forbidden idea. It appears I'm dumb and ignorant enough to have thought it was just really silly, and this reaction appears common. This is why some people find the entire incident ridiculous. I admit my opinion could be wrong, and I don't actually find it interesting enough to have remembered the details.)

Comment author: Leonhart 10 December 2010 05:26:45PM 4 points [-]

Thank you. I've found your comments very useful, not least because when younger I came uncomfortably close to being parted from a reasonable sum of money, by a group who understood the Dark Arts rather well. That was before I read Cialdini, but I'm not sure how well it would have sunk in without the object lesson.

I'm not good at thinking things are silly. That's great for getting suspension of disbelief and fun out of certain things (for example, I can enjoy JRPG plots :) but it's also a spot where one can be hit for massive damage.

As for the happy phrasing, I might have been thinking of this. (Warning: 4chan, albeit one of its nicer suburbs.)

Comment author: [deleted] 10 December 2010 03:19:52PM 4 points [-]

(I read the forbidden idea. It appears I'm dumb and ignorant enough to have thought it was just really silly, and this reaction appears common. This is why some people find the entire incident ridiculous. I admit my opinion could be wrong, and I don't actually find it interesting enough to have remembered the details.)

Same here. I think (though no one has given a definitive answer) that there is concern about the general case of the specific hypothetical incident discussed therein, not the specific incident itself.

Comment author: Broggly 14 December 2010 07:24:01PM 0 points [-]

Hmm. I only read it recently, so maybe I haven't thought through the general case enough, but I think my solution (assuming it's not totally absurd) of treating it as though it is really silly with the caveat that if it becomes non-silly I'm not exactly powerless would work for all such cases.

Comment author: Vladimir_Nesov 10 December 2010 03:25:19PM *  0 points [-]

Everyone thinks they live in [invincible mind fortresses], that they're far too intelligent/knowledgeable/rational/Bayesian/aware of their biases/expert on cults/etc to fall into cultishness. They are of course wrong, but try telling them that.

Again you tell us. Some people who think that are right. They are NOT "of course" wrong. A random person isn't guaranteed to be vulnerable, and there are people for which you can say that they are most certainly invincible. That any person is "of course vulnerable" is of course wrong as a point of simple fact.

Comment author: TheOtherDave 10 December 2010 04:15:38PM 9 points [-]

I would be interested in hearing about your evidence for the existence of people who are "most certainly invincible" to cultishness, as I'm not sure how I would go about testing that.

Comment author: David_Gerard 10 December 2010 04:10:35PM 3 points [-]

I think a lot more people are vulnerable than consider themselves vulnerable. You can substitute "most" for "all" if you like.

Comment author: Vladimir_Nesov 10 December 2010 04:19:06PM *  1 point [-]

I think a lot more people are vulnerable than consider themselves vulnerable.

I mainly object to "of course", and your argument cited here (irrespective of its correctness) doesn't even try to support it. Please be more careful in what you use, you can't just throw an arbitrarily picked affective soldier, it has to actually argue for the conclusion it's supposed to support (i.e. be (inferential) evidence in its favor to an extent that warrants changing the conclusion).

Comment author: David_Gerard 10 December 2010 04:24:14PM *  1 point [-]

I think a lot more people are vulnerable than consider themselves vulnerable.

I mainly object to "of course", and the argument I cited here (irrespective of its correctness) doesn't even try to support it.

I wasn't making an argument (a series of propositions intended to support a conclusion), I was talking about the subject in passing. These are different modes of communication, and I would have thought it reasonably clear which one was being used.

The "of course" is because it's a cognitive error: people are sure it could never happen to them. I observe them being really quickly, really certain of that when they hear of someone else falling for cultishness - that's the "of course". In some cases this will be true, but it's far from universally true. I don't know which particular error or combination of errors it is, but it does seem to be a cognitive error. It is true that I do need to work out which ones it is so that I can talk about it without those people who reply "aha, but you haven't proven right here it's every single one, aha" and think they've added something useful to discussion of the topic.

Comment author: Vladimir_Nesov 10 December 2010 04:49:46PM *  1 point [-]

I see. So they can sometimes be accidentally correct in expecting that they are not vulnerable, as in fact they will not be vulnerable, but their level of certainty in that fact will almost certainly ("of course") be off in a systematic predictable way. This interpretation works.

I wasn't making an argument (a series of propositions intended to support a conclusion), I was talking about the subject in passing. These are different modes of communication, and I would have thought it reasonably clear which one was being used.

I think of the "talking about the subject in passing" mode as "making errors, because it's easier that way", which looks to me as a good argument for making errors, but they are still errors.