My observation about cults, from personal experience leading them, is that they are a totally normal mode of human operation. People are always looking for strong leaders with vision, passion and charisma who can organize them for a larger purpose. What distinguishes a cult from a non-cult is that they are outside the norms of the mainstream society (as established by the dominant cults -- i.e. "the culture"). "Cult", "brainwashing", "deprogramming", etc. are terms of propaganda used by the dominant culture to combat competing memeplexes.
I think of cults as testbeds for new civilizations and new ways of life. In times of change, when the old ways are failing and the civilization is falling, cults may be well-positioned to expand and become the new normal. I suppose this is the memetic equivalent of marginal species who exploit mass extinctions to become genetically dominant -- cults provide memetic diversity. This is apparently what was going on in the declining years of Rome, and I see indications that something similar is happening today.
J.K. Rowling could probably manipulate Lesswrong as she sees fit by buying the site, shadowbanning all commenters, and putting up new comments using their names (but preventing the real users from seeing these) were they slowly become convinced witchcraft real.
Some old SIAI work of mine. Researching this was very difficult because the relevant religious studies area, while apparently completely repudiating most public beliefs about the subject (eg. the effectiveness of brainwashing, how damaging cults are, how large they are, whether that’s even a meaningful category which can be distinguished from mainstream religions rather than a hidden inference - a claim, I will note, which is much more plausible when you consider how abusive Scientology is to its members as compared to how abusive the Catholic Church has been etc), prefer to publish their research in book form, which makes it very hard to review any of it. Some of the key citation were papers - but the cult panic was so long ago that most of them are not online or have been digitized! I recently added some cites and realized I had not touched the draft in a year; so while this collection of notes is not really up to my preferred standards, I’m simply posting it for what it’s worth. (One lesson to take away from this is that controlling uploaded human brains will not be nearly as simple & easy as applying classic ‘brainwashing’ strategies - because those don’t actually work.)
Reading through the literature and especially the law review articles (courts flirted disconcertingly much with licensing kidnapping and abandoning free speech), I was reminded very heavily - and not in a good way - of the War on Terror.
Old American POW studies:
Started the myth of effective brain-washing. But in practice, cult attrition rates are very high! (As makes sense: if cults did not have high attrition rates, they would long ago have dominated the world due to exponential growth.) This attrition claim is made all over the literature, with some example citations being:
a back of the envelope estimate for Scientology by Steve Plakos in 2000:
Iannaccone 2003, “The Market for Martyrs” (quasi-review)
Singer in particular has been heavily criticized; “Cult/Brainwashing Cases and Freedom of Religion”, Richardson 1991:
“Overcoming The Bondage Of Victimization: A Critical Evaluation of Cult Mind Control Theories”, Bob and Gretchen Passantino Cornerstone Magazine 1994:
Gomes, Unmasking the Cults (Wikipedia quote):
“Psychological Manipulation and Society”, book review of Spying in Guruland: Inside Britain’s Cults, Shaw 1994
Anthony & Robbins 1992, “Law, Social Science and the ‘Brainwashing’ Exception to the First Amendment”:
“Brainwashed! Scholars of cults accuse each other of bad faith”, by Charlotte Allen, Lingua Franca Dec/Jan 1998: