David_Gerard comments on Build Small Skills in the Right Order - Less Wrong

90 Post author: lukeprog 17 April 2011 11:01PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (213)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukeprog 18 April 2011 02:26:53PM *  7 points [-]

Oh, you mean I should make it clear that Scientology is dangerous and people shouldn't take Scientology classes? I figured that would be obvious, but okay: I added it to the post.

Comment author: David_Gerard 18 April 2011 02:39:31PM *  12 points [-]

I'm not convinced "p.s.: don't do this thing that worked out really well for me and I shall now describe in thrilled detail" entirely makes it no longer functionally a personal recommendation, but it's possibly better than nothing. Thank you.

Comment author: Apprentice 18 April 2011 03:06:39PM 7 points [-]

Yes but LessWrong is a lot like this - witness all the discussions in thrilled detail of drugs that put your brain into a more effective/enjoyable state. It's assumed that the readership is intelligent/responsible enough to handle this sort of thing.

The desire to succeed in unorthodox ways ("cheat" at life) is strong in many members of this community - Luke's Scientology story fits that pattern very well. It certainly makes me want to try a com course and I've read about Scientology in endless detail - including some of your work.

Comment author: wnoise 18 April 2011 05:43:56PM 8 points [-]

Yes but LessWrong is a lot like this - witness all the discussions in thrilled detail of drugs that put your brain into a more effective/enjoyable state. It's assumed that the readership is intelligent/responsible enough to handle this sort of thing.

The outside culture has enough warnings about dangers of using drugs that we don't have to repeat them here. Everybody knows that playing with them can fry your brain, and you should take proper precautions. I don't think the outside culture has enough warnings about psychological manipulation techniques in general, nor this particular sect. People routinely think they'll be less influenced than they are.

Comment author: Risto_Saarelma 18 April 2011 06:45:17PM 20 points [-]

And there's also the thing that while the people who hang around at LW probably have more ammo than usual against the overt bullshit of cults, they also might have some traits that make them more susceptible to cult recruitment. Namely, sparse social networks, which makes you vulnerable to a bunch of techniques that create the feeling of belonging and acceptance of the new community, and tolerance of practices and ideas outside the social mainstream, which gets cult belief systems that don't immediately trigger bullshit warnings inside your head.

The Aum Shinrikyo cult in Japan that did the subway sarin gas thing reportedly recruited lots of science and engineering students. An engineering mindset will also keep you working from the internalized bullshit against social proof, since science and engineering is a lot about about how weird stuff extrapolated beyond conventional norms works and gives results.

tl;dr: You're not as smart as you think, probably have a mild mood disorder from lack of satisfactory social interaction, and have no idea how you'll subconsciously react to direct cult brainwashing techniques. Don't mess with cults.

Comment author: [deleted] 18 April 2011 07:00:33PM 6 points [-]

How about a word on the major religions? The most obvious difference between a cult and a religion is that the religion is many orders of magnitude more successful at recruitment - which is the very thing that we are being warned about with respect to cults.

Comment author: Risto_Saarelma 18 April 2011 07:06:47PM 29 points [-]

Parasite species that have been around a long time have mostly evolved not to kill their host very fast. With new species, all bets are off.

Comment author: David_Gerard 18 April 2011 08:17:50PM 9 points [-]

The Mormons are a good comparison. They were dangerous lunatics in the mid-1800s - and Brigham Young was a murderous nutter on a par with David Miscavige. These days, they're slightly weirdy but very nice (if very, very conservative) people; good neighbours.

Comment author: [deleted] 18 April 2011 07:19:46PM 3 points [-]

You must mean "kill off" metaphorically, since I don't recall any incidents in which Scientology has killed off Scientologitsts. In contrast I can recall many very recent incidents in which one old religion - Islam - has killed off adherents. But if "kill off" is a metaphor, then what is the literal danger from Scientology which is being referred to metaphorically as "kill off the host"?

Comment author: Risto_Saarelma 18 April 2011 07:51:19PM *  6 points [-]

Ruin their life or mess them up mentally.

Comment author: David_Gerard 18 April 2011 08:15:39PM *  8 points [-]

http://en.wikipedia.org/wiki/Death_of_Lisa_McPherson - and she was hardly the first.

I would caution against using "I don't recall" to mean "I haven't researched even slightly".

Comment author: [deleted] 18 April 2011 08:28:31PM 1 point [-]

I would caution against using "I don't recall" to mean "I haven't researched even slightly".

I used "I don't recall" to mean "I don't recall". Go ahead and bash me for failing to research the question but please don't put your words and ideas in my writing.

Comment author: TimFreeman 19 April 2011 01:00:35AM *  0 points [-]

Check out Auditing Procedure R2-45. There are also a number of less formal murders attributed to them. Ask Google for "Scientology Murder".

Comment author: bogus 19 April 2011 01:28:32AM *  0 points [-]

Please do not use value-laden and unsupported terms such as "murder" here. Yes, there are some cases of controversial deaths involving Scienology, but none of these could be described as murder of either the formal or less-formal sort.

The existence of R2-45 is rather unsettling, but apparently this 'auditing procedure' has never been enacted.

Comment author: faul_sname 13 May 2012 07:43:53PM 1 point [-]

Growth/attrition rates are actually the thing to look at here. Scientology is faster-growing than just about any other modern religion, though the attrition rate is also very high. In order to figure out virulency, figure out what population the S-curve of members of that religion will top out at. If growth is slowing, you're almost there. If growth is steady, you're about halfway there. If growth is exponential or approximately so, you're looking at a religion in its infancy.

Comment author: David_Gerard 18 April 2011 08:40:54PM 2 points [-]

The Aum Shinrikyo cult in Japan that did the subway sarin gas thing reportedly recruited lots of science and engineering students. An engineering mindset will also keep you working from the internalized bullshit against social proof, since science and engineering is a lot about about how weird stuff extrapolated beyond conventional norms works and gives results.

This has of course been covered here before (with reference to this and this).

Comment author: Swimmer963 26 April 2011 11:16:35PM 1 point [-]

probably have a mild mood disorder from lack of satisfactory social interaction

Umm. Not all of us. I may be vulnerable to cults for other reasons, namely my conformist personality, but not lack of people to talk to.

Comment author: katydee 26 April 2011 11:13:32PM *  1 point [-]

"You probably have a minor mood disorder from lack of satisfactory social interaction" seems like a rather harsh description of the members of this community. What data generated that thought?

Comment author: gwern 27 April 2011 01:34:40AM 7 points [-]

I agree with the description. Why? Because the joy people describe at going to the meetups seems out of proportion to what goes on in the meetups - unless, as the old saying goes, hunger is the best spice.

Comment author: Risto_Saarelma 27 April 2011 04:57:28AM *  5 points [-]

I started with the assumption that most people posting here live alone or with a small immediate family and occasional interaction with acquaintances instead of as a part of a tightly knit tribe of some dozens of people who share their values and whom they have constant social interaction with. Then thought what the probable bias for site members to belong into a mainstream society tribe-equivalents like churches, sports fan groups, gangs or political organizations was.

The "mood disorder" thing is hyperbole for "your brain would like to be in a more tribe-like social environment than it is in now", not an attempt at a clinical diagnosis.

Comment author: handoflixue 26 April 2011 10:58:42PM 1 point [-]

Oddly, a "sense of belonging" usually makes me feel alienated and uncomfortable. It's the rare exceptions like LessWrong, where it actually feels like I do fit, and am being challenged and growing and free to express myself, that avoid that.

Comment author: Alicorn 26 April 2011 11:10:26PM 3 points [-]

Oddly, a "sense of belonging" usually makes me feel alienated and uncomfortable.

This sounds very odd. In fact, it sounds oxymoronic. Can you explain?

Comment author: AdeleneDawner 26 April 2011 11:21:14PM *  12 points [-]

I can take a shot at it, having experienced something similar.

The general situation usually follows the pattern of "There is a group with easily-noticeable standards A, B, and C and less-easily-noticeable standards X, Y, and Z. I conform to A, B, and C (though probably for different reasons than they do), but not to (some subset of) X , Y, and Z, but since X and Y and Z don't come up very often, 1) they haven't figured out that I don't fit them, and 2) I didn't realize that those standards were significant until after I'd been accepted as a member of the group (which is where the 'sense of belonging' comes in). At no point did I actually mislead the group with regards to X, Y, or Z, but it's very likely that if they find out that I don't conform to them, they will assume that I did and there will be large amounts of drama."

This usually leads to an inclination to hide facts relating to X, Y, and Z, which feels from the inside like being alienated and uncomfortable.

ETA: This isn't necessarily something that a person would have to be consciously aware of in order for it to happen, and it can also be based on a person's assumptions about X/Y/Z-like standards if a given group doesn't make them explicit.

Comment author: handoflixue 26 April 2011 11:39:39PM *  4 points [-]

Adelene's response strikes me as a similar experience. I should also admit that I'm having a lot of trouble actually getting a concrete description of the experience, as it's primarily emotional/subconscious, but here's my own go at it:

I suppose the short version is that while I have the social/emotional response of "belonging and acceptance", I don't actually feel safe relaxing and letting down my guard around those groups, which produces a secondary emotional response of feeling alienated and uncomfortable that I have to keep those defenses up.


There are various social behaviors that groups will exhibit to build a very strong "sense of belonging", and it's more an emotional evaluation than an intellectual one - although the other part is that I often 99% fit with a group, am clearly a valuable member of the group, and risk getting expelled if I reveal that other 1% of myself.

More specifically, I belong to a few groups where revealing one's status can still result in fairly sharp social ostracization . Thus, once I've found a group where I "belong", I run in to the choice of risking all of that to be accepted "for who I really am", or just shutting up and keeping quiet about things that almost never come up anyways.

In the case of LessWrong, I feel safe because the community strikes me as much more likely to be tolerant of these things, because an online community has much less power to hurt me, and because these things are extremely unlikely to come up here to begin with (and, being an online forum, I can devote time to carefully crafting posts not to reveal anything; that's still annoying, but gets written off as "I don't want to post publicly about this" rather than "LessWrong is unsafe")


The other aspect is simply that a lot of standard recruitment/retention techniques trigger a visceral aversion to me, even if I don't view the group as a threat and genuinely do want to be a member.

Comment author: NancyLebovitz 22 July 2012 02:52:18PM 1 point [-]

I've got a streak of that, though of a different flavor. Some types of ceremonial efforts to solidify group cohesion don't work for me, so I feel alienated from any group where there's an assumption that I'll feel good and devoted because of enforced symbolism.

To be less abstract about it all, I'm American, whatever that means. I can be defensive and even mildly jingoistic about America (though I consider the latter a failing)-- but I'd be a lot more comfortable with the place and the identity if it weren't for all the damned flags.

In other news, I've been wondering lately whether it would be closer to the truth if, instead of thinking of myself as Jewish (ethnically), it would be better to frame it as "People kept telling me I was Jewish until I started believing it".

Comment author: shminux 22 July 2012 07:37:32PM *  1 point [-]

I can be defensive and even mildly jingoistic about America

The US has one of the most effective brainwashing systems in the (first) world, patriotism-wise. I suspect that a part of it is the historical narrative of a real or imagined success against formidable odds, all in the last 200 years or so. The message "America is great" is also constantly all over the school system and the media. This is really hard to resist, no matter how often you repeat to yourself "I ought to keep my identity small".

I've been wondering lately whether it would be closer to the truth if, instead of thinking of myself as Jewish (ethnically), it would be better to frame it as "People kept telling me I was Jewish until I started believing it".

I heard that sentiment many times, not necessarily from people of Jewish descent, although the latter are an easy example. Jews in the early 20th century Germany thought if themselves as Germans, until "real Germans" disabused them of that notion in 1930s. Same happened in Russia in 1950s. Various Yugoslavian ethnicities suddenly realized in 1990s that they were not just Yugoslavians, but Serbs, Albanians, Croatians etc., and those who did not were quickly and forcefully reminded of it by their neighbors.

Comment author: Gray 27 April 2011 02:15:58AM 0 points [-]

I somewhat relate to his comment, and for me it's because of how much persona, holding myself back, and not letting myself go it requires to be accepted by others. When, and if, it actually does work, it feels like here all I was trying to do was be a nice guy, and now the ruse worked? Now it's like you've committed yourself to it.

Comment author: Eugine_Nier 18 April 2011 06:55:28PM 1 point [-]
Comment author: Risto_Saarelma 18 April 2011 07:03:22PM 8 points [-]

He might not. But things will be in his favor if you go in thinking knowing physics and science will make you impervious to the dark arts, without knowing a lot about psychology, cult and influence techniques and the messier stuff inside your own head.

(I'm not sure if you want to say something extra here by quoting a thing that was described as the "second most dangerous dark side meme" in the linked comment.)

Comment author: Gray 19 April 2011 04:15:57AM 2 points [-]

I wonder about this idea that knowing how someone will be manipulating you is any defense at all from being manipulated by that person. It sounds plausible, but is there any evidence at all that knowledge can have this affect?

Or is knowledge not wholly intellectual, and can be considered a species of manipulation, but not manipulation of the dark arts variety. Maybe even "light arts manipulation"? Sorry, had to throw this last paragraph in there because I thought it was interesting.

Comment author: Risto_Saarelma 19 April 2011 05:16:30AM *  6 points [-]

Compare "I've only known this guy for half an hour, but he seems really likable" and "I've only known this guy for half an hour, he's been running through the tricks from the cult salesman playbook and is giving off a real likable impression at this point".

You still need to have your own head game in order to actually counteract the subconscious impressions you are getting, but it will probably help to know that a contest is even happening.

Comment author: Gray 19 April 2011 03:34:47PM 1 point [-]

I think what you say is plausible. But I also think that it is also plausible that a "likable impression" isn't just an appearance, but the effect of you actually starting to like the guy. I think that's the sort of thing that concerns me, that at a certain point our social instincts take over and we lose the ability to detach ourselves from the situation.

Comment author: CuSithBell 20 April 2011 12:20:26AM *  1 point [-]

Interesting. My intuition and experience say this is screamingly, overtly incorrect. The fact that yours do not means I'm probably wrong - either about the 'overtly' or the 'incorrect'!

Comment author: David_Gerard 18 April 2011 08:41:26PM 1 point [-]

But things will be in his favor if you go in thinking knowing physics and science will make you impervious to the dark arts, without knowing a lot about psychology, cult and influence techniques and the messier stuff inside your own head.

I do believe you've nailed it. Well done, sir.

Comment author: faul_sname 13 May 2012 07:39:23PM 0 points [-]

You're not as smart as you think, probably have a mild mood disorder from lack of satisfactory social interaction, and have no idea how you'll subconsciously react to direct cult brainwashing techniques. Don't mess with cults.

This is an important point. If you do mess with cults, start with the more innocuous ones before you face the heavy guns. Make sure you can resist the community in an average church before you test yourself against Scientology.

Comment author: NancyLebovitz 22 July 2012 03:08:11PM 0 points [-]

One of the impressive things about Sufism (at least as described by Idris Shah) is that they wouldn't take people as students who didn't already have social lives.

Comment author: David_Gerard 18 April 2011 08:39:02PM *  0 points [-]

Arguably, Internet culture has a tremendous amount of information on the dangers of Scientology in particular. (And I'm one of the people who put it there personally.) But you are entirely correct: people are convinced they're much less manipulable than they are. I need to write something for LW on the subject (as I've been idly contemplating doing for about 6 months).

Comment author: jasticE 19 April 2011 09:00:51AM 1 point [-]

Do you know of any techniques to measure your own manipulability somewhat objectively?

Comment author: handoflixue 26 April 2011 11:14:44PM 3 points [-]

I would think the easiest method, albeit not terribly objective, would simply be to get someone who is fairly good at manipulation and play out scenarios with them. I've done this a few times as the manipulator, and it's sort of scary how easily I can manipulate people in specific games, even when they know the rules and have witnessed some of my techniques.

If you do try it, I'll comment that time and social pressure help me a lot in making people more pliable, too. I do these as a group exercise, so there's a lot of peer pressure both to perform well, and not to use exactly the sort of "cheats" you should be using to resist manipulation. It's also helped that I've always known the group and thus known how to tweak myself to hit specific weaknesses.

If you find something more useful than this, I'd love to hear it. I've merely learned I'm fairly good at manipulating - I have no clue how good I am at resisting :)

Comment author: NancyLebovitz 27 April 2011 11:25:57AM 1 point [-]

time and social pressure help me a lot in making people more pliable

That reminds me of a bit from a book about art forgery-- that need, greed, and speed make people more gullible.

Comment author: cousin_it 27 April 2011 10:13:05AM 1 point [-]

I'd love to try this (being the manipulatee). Do your mind tricks work over Skype?

Comment author: handoflixue 29 April 2011 07:51:23AM 0 points [-]

Having not tested them, I wouldn't be sure. I tend to do best with people who are either following an easily inferred pattern (office workers, security, etc.) or people who I know personally, which would make it harder to do with someone I don't know. You also are neither "disposable" (someone I'll never deal with again) nor a friend, which adds a bit of social awkwardness.

Given that's an entire paragraph of excuses, I suppose I should offer to try it anyway, if you want :)

Comment author: cousin_it 29 April 2011 08:53:16AM 0 points [-]

Good! How about Sunday evening (CEST)?

Comment author: David_Gerard 19 April 2011 03:50:06PM *  1 point [-]

I have occasionally seen quizzes that purport to tell you how biased you are in purportedly relevant ways to cult susceptibility. I can't say I found any of them revelatory, as, since you know what the test is testing, it's way too easy to answer with the right answer rather than the true readout, even when you want the latter. I suppose proper testing would have to be similar to psychological measures of cognitive biases.

Comment author: David_Gerard 18 April 2011 03:11:17PM *  11 points [-]

Sewer-diving could be fun, and instructive! But a note or few about adequate preparation first strikes me as a really good idea. Particularly when the story turns out to be "and then I swallowed this sample of engineered resistant mycobacterium tuberculosis, and I felt great." Hubris is one of the dangers of a little knowledge.

Comment author: Clippy 20 April 2011 07:09:55PM *  2 points [-]

Sewer-diving is, in fact, fun and safe for humans, and your warnings about the dangers are alarmist and excessive.

Scientology classes are also safe.

Comment author: AdeleneDawner 20 April 2011 07:11:26PM 6 points [-]

How did you come to the conclusion that this was a good comment to post?

Comment author: Clippy 20 April 2011 07:14:03PM 1 point [-]

How did you come to the conclusion that the parent of the comment containing this sentence was a good comment to post?

Are you attempting to direct me on an endlessly-recurring chain of justification? At some point, reflection must stop and action must be taken, or else you will use up all free energy and entropize just thinking of your next action. Correct reasoning teaches you this very quickly.

Comment author: AdeleneDawner 20 April 2011 08:10:46PM 3 points [-]

How did you come to the conclusion that the parent of the comment containing this sentence was a good comment to post?

By heuristic based processing, as with how I do most things. It seems reasonable to assume that the same isn't true of you, though, so I expected a rather more useful answer to my question. (Relevant heuristics include 'if confused, ask for information' and 'alert friend-type people to mistakes so that they can avoid those mistakes in the future'.)

Are you attempting to direct me on an endlessly-recurring chain of justification? At some point, reflection must stop and action must be taken, or else you will use up all free energy and entropize just thinking of your next action. Correct reasoning teaches you this very quickly.

I wasn't, actually. I suspect that whatever system you used to decide to make that post is poorly calibrated, and intended to offer help in debugging it. It's also possible that my model of you is not as accurate as it could be, and that's what needs debugging. In either case, gathering more information is a reasonable early step in the process.

Comment author: Clippy 20 April 2011 08:42:13PM 2 points [-]

By heuristic based processing, as with how I do most things. It seems reasonable to assume that the same isn't true of you, though, so I expected a rather more useful answer to my question. (Relevant heuristics include 'if confused, ask for information' and 'alert friend-type people to mistakes so that they can avoid those mistakes in the future'.)

I also use heuristic reasoning, (governed by the meta-heuristic of correct reasoning), and here I thought that User:David_Gerard was significantly overstating the risks of sewer-diving and Scientology classes for humans. Therefore, I added my "independent component" to the discussion.

Comment author: David_Gerard 20 April 2011 07:15:12PM 0 points [-]

Sewer diving is in fact a favourite of urban explorers. And I must admit that trolling Scientology in my dissolute youth was lots of fun :-D