Review

The start of the post is copy-pasted below. Note that the post is anonymous, and I am not claiming to have written it.


Some people in the rationalist community are concerned about risks of physical violence from Ziz and some of her associates. Following discussions with several people, I’m posting here to explain where those concerns come from, and recommend some responses.

TLDR (details and links in the post body)

  • Over the past few years, Ziz has repeatedly called for the deaths of many different classes of people.
  • In August of 2022, Ziz seems to have faked her own death. Ziz’s close associate Gwen Danielson may have done the same thing in April of 2022.
  • In November of 2022, three associates of Ziz (Somnulence “Somni” Logencia, Emma Borhanian, and someone going by the alias “Suri Dao”) got into a violent conflict with their landlord in Vallejo, California, according to court records and news reports. Somni stabbed the landlord in the back with a sword, and the landlord shot Somni and Emma. Emma died, and Somni and Suri were arrested. Ziz and Gwen were seen by police at the scene, alive.
  • I gather that Pennsylvania police believe, based on a lot of circumstantial evidence and investigation, that one or more of Ziz, Michelle “Jamie” Zajko, or Daniel Blank was very plausibly involved in the murder of Jamie’s parents in Pennsylvania around December 31st, 2022.
  • Ziz is currently in police custody on charges related to obstructing a PA police investigation. Daniel, Jamie, and Gwen are not in custody (as far as I know), and I don’t know their locations.
  • I don’t know of any concrete plans by Ziz or her associates to do anything else violent, but it seemed like a good idea to raise a loud alarm about all of this now. People should speak up (or at least tell a friend) if they have information, and should take whatever safety precautions make sense for their situation in case more violence occurs at some point in the future.

Even seemingly minor pieces of information might be helpful here, since they could add up to a clearer picture when combined with other information that also seems minor. If you want to share information privately, you can email me at sefashapiro@gmail.com, or submit information anonymously through this form.

This is a complicated situation that I don’t fully understand, and it’s likely that I’m getting some facts wrong. I’ve talked to a lot of people in an attempt to piece together what happened, and I’ll try to update this post with corrections or important details if they’re brought to my attention.

Please keep in mind, as I am trying to keep in mind myself, that every observation is evidence for more than one hypothesis, that things are often not what they seem, and that it’s useful to make the effort to think about both what’s appropriate to do in the world where your best guesses are true, and what’s appropriate to do in the world where your best guesses are wrong. Split and commit is better than seeking confirmation of a single theory.

[Link] A community alert about Ziz
New Comment
132 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

The possible involvement of Daniel Blank is the most surprising to me. As the linked article notes, Daniel seems like about the least likely person to commit a homicide. I've not had contact with him in a few years, but I recall Daniel as sensitive, serious, and caring, and a person who would go out of his way to avoid causing someone else harm. I don't know what's happened in the intervening time to lead to his involvement here.

Part of the process that Ziz seems to advocate involves inducing alternative personalities in people. The process that unleashed the Maia personality in Chris Pazek led in Ziz's words to their suicide. 

This process could easily lead to a personality being active in a person that has very different characteristics among various axis than the personality you knew years earlier.

"Least likely person to do X" is an interesting concept. People who deal with a shadow that has an attribute they despise often overcompensate to fight that shadow. That shadow might provide a substance based on which the alternative personality that acts very differently can be built.

"Least likely person to do X" is an interesting concept. People who deal with a shadow that has an attribute they despise often overcompensate to fight that shadow. That shadow might provide a substance based on which the alternative personality that acts very differently can be built.

I feel complicated about this; in a world where everyone is doing split-and-commit (or something similar) by default, this feels straightforwardly good to note.

But like. In this world, this feels like fodder for "you should interpret evidence that someone is unusually unlikely to X as evidence that they're actually quite likely to X, given some trigger."

And I think this is the sort of thing that is true in some very small minority of cases and an outright (and false) weapon all the other times.

I think we should be real careful about interpreting evidence against X as evidence of X, even given that the psychological structure you just described is one that exists.

I do agree that it's not an easy subject to speak about. 

There are two separate issues at hand: 

  1. What happens if you have multiple personalities
  2. Suppressed anger and its effects 

Sometimes abused children develop multiple-personality disorders where they develop new personalities to deal with situations where the characteristics.

I have one friend who has the multiple-personality disorder and is able to switch between them. According to him, he often faced situations where a person was used to dealing with one of his personalities and really surprised when he switched to another personality that has different personality characteristics. 

The technique that Ziz advocates seems to intentionally created something that's similar to multiple personality disorder and I would expect that the same pattern hold where at least one of the personalities is able to do things that are far outside of what the old unified self would do. 

I think good evidence that someone is unlikely to cause harm is about observing that they have functional strategies for taking care of boundary violations. If someone's reaction is instead to ignore their own needs, repress any anger that come... (read more)

6Edmund Nelson
That's fair from a mental POV, but Daniel Blank physically has poor hand-eye coordination and bad reflexes, meaning that if he tried to shoot somebody he'd be extremely likely to miss at all but the shortest of ranges. (the murder in question was definitely gunshot based)   While he could be a part of the conspiracy his ability to have physically committed the actual act is questionable. 
3Viliam
Is this an empirical question? Like, could you gather 1000 people, measure how violent they are, then somehow summon their shadows, and measure again? My guess would be that the shadow is on average more violent than the original person, but it is not true that less violent person = more violent shadow. (I would probably expect something like: each shadow gets a bonus +5 on violence compared to the original person; including people who already started very violent.)

The process that unleashed the Maia personality 

I think that this misidentifies the crux of the internal argument Ziz created and the actual chain of events a bit. 

imo, Maia was trans and the components of her mind (the alter(s) they debucketed into "Shine") saw the body was physically male and decided that the decision-theoretically correct thing to do was to basically ignore being trans in favor of maximizing influence to save the world. Choosing to transition was pitted against being trans because of the cultural oppression against queers. I've run into this attitude among rationalist queers numerous times independently from Ziz and "I can't transition that will stop me from being a good EA" seems troubling common sentiment.

 Prior to getting involved with Ziz, the "Shine" half of her personality had basically been running her system on an adversarial 'we must act or else' fear response loop around saving the multiverse from evil using timeless decision theory in order to brute force the subjunctive evolution of the multiverse. 

So Ziz and Paseks start interacting, and at that point the "Maia" parts of her had basically been like, traumatized into submission an... (read more)

9Eli Tyre
Can you share? I would like to have a clearer sense of what happened to them. If there's info that I don't know, I'd like to see it.

things i'm going off:

the pdf archive of Maia's blog posted by Ziz to sinseriously (I have it downloaded to backup as well)
the archive.org backup of Fluttershy's blog
Ziz's account of the event (and how sparse and weirdly guilt ridden it is for her)
several oblique references to the situation that Ziz makes
various reports about the situation posted to LW which can be found by searching Pasek

From this i've developed my own model of what ziz et al have been calling "single-good interhemispheric game theory" which is just extremely advanced and high level beating yourself up while insisting you're great at your emotions. There is a particular flavor of cPTSD that seems disproportionately overrepresented within the LW/EA community umbrella, and it looks like this:

hyperactivity
perfectionist compulsion to overachieve
always-on
constantly thinking with a rich inner world
high scrupulosity blurring into OCD tendencies
anxiety with seemingly good justifications (it's not paranoia if...)
an impressive degree of self-control (and the inability to relax fully)
catastrophizing
dissociation from the body

this is a mode of a cPTSD flight response. Under the cPTSD model, "Shine" could be thought of as a toxi... (read more)

2Eli Tyre
What do you mean by this? Like, she's better than average at predicting people's behavior in various circumstances?  
2Slimepriestess
Less predictive and more observational, but sorta yeah? Like, if someone is lying to themselves and playing all these weird internal denial/repression games internally, there are tells for that which you can learn to notice. After a while it gets pretty obvious what the behaviors you observe in someone actually mean (vs what they say those behaviors mean). Why I say "uncomfortably so" is that speaking from my own experiences, once you learn to read people this way, it's not really something you can turn off again. That can add a lot of friction to social interactions, where it seems like everyone is just constantly trying to bullshit you.
8ChristianKl
Do you have any indication that Pasek was trans before they spoke with Ziz? Pasek couch-surfed at my place for a few days around a LessWrong Community weekend he attended and we had deep conversations then. I think that was 1-2 years before he got into contact with Ziz. At that time he was using heavy optimization pressure on himself. In my memory, he had some logging where he wrote something every hour to measure his productivity. He was also heavily into timeless decision theory-based utilitarian consequentialism at that time.  I'd buy it that there was an internal conflict at the time. I believe that process that Ziz proposed took that internal conflict and create the Shine and Maia personalities out of them.  If a person is putting an extraordinary amount of effort into being nice (which is what Gordon Seidoh Worley observed) there's likely an internal conflict. What Ziz is doing allows transforming that internal conflict into two parts, one that's very nice and one that's opposed to being nice.   Usually, people who do that have a lot of akrasia. Pasek is different in that they managed not to have that. Most people would be blocked by internal friction from doing the kind of productivity optimization that Pasek did.  I think that Pasek was smart enough to know that it's good to give the part of him that "i am a creature that exists in a body. I have needs and desires and want to be happy and feel safe"  things to satisfy it. That part wanted to be happy, so they did some body work intervention to feel happy (and wrote on the blog that someone that didn't solve the issue and that people aren't really seeking happiness). That part wanted that they identified themselves publically as Maia, so they did that. That part wanted to transition, so they took hormones.  Shine did find a justification. The way they committed suicide however was not done in a way that sounds like TDT was involved. They could have written an actual suicide note to the people that care

Random fact: I just assumed Pasek was trans, based on some vague writing style / username, and then one day met with them on Google Hangouts to discuss some AI stuff, and they brought up that just yesterday they had decided/realized they were female. I brought up "uh, oh, I had just kinda assumed you were a trans woman this whole time" and they said "well, bayes points to you I guess".

I separately knew Ziz but I think didn't know there was a connection there at the time.

4ChristianKl
Do you know when that was? According to Ziz the connection in the beginning used to be that they exchanged emails. From speaking to one of Pasek's roommates about his death, there was zero mention of Ziz so I don't think Pasek talked to other people about his connection to Ziz. 
6Raemon
Early 2018
5ChristianKl
According to Ziz's timeline her first contact with Pazek was after Pasek commented on her blog in Dezember 2017. Looking through messages from Pazek for his writing style, there's a certain kind of positivity in their writing style. He wrote Ziz at the time You say all the right things! I cannot marry you right now but let’s be best friends forever.  In June 2016 Pasek wrote me "I've recently been doing a little project in which I talk with random LW users on Skype. (This has turned out to be a lot of fun!) So if you feel like it and have the time - please let me know and we an arrange to talk sometime.", so that behavior seems to be older.  This is an untypical style of communicating and there are likely some transwomen who choose that style to signal their feminity.  Pasek didn't adopt it for that signaling purpose. At the same time factors that pointed toward trans might also made that communication style attractive to him. Another explanation for him communicating like that is that it's what he learned to fit-in in Japan where a lot of politeness is called for. His conscious analysis also likely pointed into the direction of this style being useful for connecting with people.
-1Edmund Nelson
Knowing what I know about Daniel, I could easily psychologically manipulate him into doing terrible acts if need be. I can easily see a person who is a master of being a cult leader (which Ziz is a top tier cult leader) mentally breaking Daniel. He (at least was) a socially awkward person who took things extremely literally and was easy to push around. He's definitely up there on the "easy" category, and cult leaders like Ziz need somebody like that. Going over Ziz's tactics, while Daniel is unlikely to have developed multiple personalities, he'd easily enter a few of the states Ziz mentions.     However I have no clue how Daniel would physically be able to commit certain acts his hand eye coordination is bad, and he's physically not very strong. If he tried to knife me to death he'd get knocked out by my overhand right or heel hooked into submission well before he got enough stabs in (I'm not a great boxer), 

I imagine him being manipulated into giving them a place to live. I have trouble imagining him being manipulated into conspiring towards a specific goal of murder. I wouldn't be surprised if the police care little for such a distinction.

-2Dagon
Religion is a helluva drug.  

AFAIK most religious people don't start suddenly committing homicide. It more seems like the relevant factor is specific to Zizianism, which appears to compel people who adopt it to take extreme actions, even if they are sensitive, serious, caring, and would go out of their way to avoid causing someone else harm.

Or perhaps, especially if they are like that? I had interacted with somnilogical a while ago, and I was quite suprised that ey had joined Zizianism as that was not my impression of the kind of person ey were. But I didn't interact hugely much with somni so I may have misjudged em.

[-]TAG128

People can do bad things if they think something much larger is at stake. Some but not all religions suggest that. Some non religious philosophies do as well, notably utilitarianism. Many, but not all, utilitarians solve the problem by not fully believing in it. Zizians adopt extreme utilitarianism about animal welfare.

-44[anonymous]
-23Richard_Kennaway
[-]Raemon2817

One thing I'm interested in more info on, not quite related to the OP, is whether SquirrelInHell/Pasek's other mental techniques listed on bewelltuned.com are basically good, or are hazardous in some way. If there are people in this thread who've dug into the details here, or tried their Motor Cortex techniques, I'd like to hear about that.

Their Tuning Your Cognitive Strategies post is one of the most impactful techniques I've tried, that I highly recommend. When I recommend it, people periodically ask "wait, isn't this the person who committed suicide after interacting with Ziz? How do you know their techniques are safe?"

Their Cognitive Strategies and Emotional Processing posts seem extremely straightforwardly useful, and pretty straightforwardly not-harmful to me. I haven't tried their muscle cortex stuff because it seemed both more effortful and less obviously useful. I could imagine it turning out to also be straightforwardly good, or I could imagine in veering into weirder territories.

They stated that bewelltuned.com was where they put stuff that they were confident was safe/good, but, idk how much stock to put in that.

I've read everything from Pasek's site, have copies of it saved for reference, and i use it extensively. I don't think any of the big essays are bad advice, (barring the one about suicide) and like, the thing about noticing deltas for example, was extremely helpful to me. I also read through her big notes glossary document in chronological order (so bottom to top) to get a general feel for the order she took in the LW diaspora corpus. My general view though is that while all the techniques listed are good that doesn't stop you from using them to repress the fact that you're constantly beating down your emotions, and getting extremely good at doing that by using advanced mental hacking techniques just made the problem that much worse. Interestingly, early Ziz warns about this exact thing. bewelltuned in particular, while being decent content in the abstract, does seem particularly suited to being used to adversarially bully your inner child.

3Said Achmiz
That website says: Are we to understand from this that the author of this document learned the skills, was consequently certain that the skills worked and improved their life, and then… committed suicide? It seems clear that an epistemic error was made somewhere in that progression.
[-]Raemon2013

Sure, but, like, people can discover things like "exercise is good for you" or "eating healthy is good for you", and nonetheless have unrelated problems that cause them to commit suicide (they can even end up committing suicide due eating-disorder-related reasons), so, sure an error happened somewhere, but that's not very strong evidence that the error is related to having discovered and got excited about exercise being good for you and eating healthy being good for you.

(Tuning your cognitive strategies paid off for me immediately, in a straightforward way that made sense)

I think there's a thing where people with a lot of mental problems tend to get very enthusiastic about various therapy-type techniques and genuinely get a lot of benefit out of them. But then they still have massive problems because their original problems were so humongous to begin with, even if they improve a lot they're still worse on mental health than the median person. (This has historically also described me, even though I have also been getting better over the long term.)

Even if they are genuinely making steady progress, that progress might not be fast enough to ensure that an unexpected shock or a set of adverse consequences won't bring them down.

2Adam Zerner
Local Validity as a Key to Sanity and Civilization is related.
[-]lc27173

What the fuck

Conflict (especially about whether one should breed animals to have extreme bodies and then lock them up in tiny unsanitary cages so they can grow up and then slaughter and eat them) + a strong commitment to/philosophy of escalating all the way for incentive reasons are the main ingredients I think.

[-]Ratios11-5

I don't think it's that far-fetched to view what humanity does to animals as something equivalent to the Holocaust. And if you accept this, almost everyone is either a nazi or nazi collaborator.

When you take this idea seriously and commit to stopping this with all your heart, you get Ziz.

[-]lc6280

When you take this idea seriously and commit to stopping this with all your heart, you get Ziz.

No, you don't, because Ziz-style violence is completely ineffective at improving animal welfare. It's dramatic and self-destructive and might express soundly their factional leanings, but that doesn't make it accomplish the thing in question.

Further, none of the murders & attempted murders the gang has committed so far seem to be against factory farm workers, so I don't understand this idea that Ziz is motivated by ambitions of political terrorism at all. Reading their posts it sounds more like Ziz misunderstood decision theory as saying "retaliate aggressively all the time" and started a cult around that.

2Ratios
In both cases, the violence they used (Which I'm not condoning) seemed meant for resource acquisition (a precondition for anything else you must do). It's not just randomly hurting people. I agree that it seems they are being quite ineffective and immoral. But I don't think that contradicts the fact that she's doing what she's doing because she believes humanity is evil because everyone seems to be ok with factory farming. ("flesh-eating monsters") This is a strawman.

While "retaliate aggressively all the time" does seem like a strawman, it is worth noting that Ziz rejects causal decision theory (a la "retaliate aggressively if it seems like it would cause things to go better, and avoid retaliating if it seems like it would cause things to go worse") in favor of some sort of timeless/updateless decision theory (a la "retaliate aggressively even if it would cause things to go worse, as long as this means your retaliation is predictable enough to avoid ever running into the situation where you have to retaliate").

Meanwhile other rationalist orgs might pretend to run on timeless/updateless decision theory but seem in practice to actually run on causal decision theory.

For an example, see the "rationalist fleet" post.  Among other conflicts, it describes getting into a drawn-out conflict with a roommate/subletter (who by Ziz's account was pretty abusive), ending with the below; it seems pretty illustrative of Ziz's thought-process (and has nothing to do with veganism):

We all had reports to make to CPS. We called the landlord. The nanny reported him for driving drunk to Uber. I went to the police again, showed them my bruise, they still said I couldn’t prove anything. I thought I had a deontological obligation not to let him profit by aggression meant to drive me out of my home for resources. I wondered if this was enough. I felt like maybe I was deontologically obligated to stay there, but, fuck. The door didn’t really close anymore. There was a hole in it. I heard his child was taken away, and was satisfied with that. Then I heard he got him back. I considered whether to show up at fuck o’clock in the morning and put something in his car’s gas tank to destroy it. Murphyjitsu: bring a charged cordless drill to create a hole if it was one of those gas tank caps that locked, and actually look up what things will destroy an engine. (Not done wi

... (read more)
[-]lc2218

Meanwhile other rationalist orgs might pretend to run on timeless/updateless decision theory but seem in practice to actually run on causal decision theory.

What semi-inteligent humans natively do without thinking all that hard is closer to "updateless" decision theory than causal decision theory, and people who think that fancy decision theories imply radically different optimal behavior on the part of regular people are usually gravely misunderstanding what they actually say. The Zizians are an example of this.

[-]lc152

In both cases, the violence they used (Which I'm not condoning) seemed meant for resource acquisition (a precondition for anything else you must do).

This is such an unrealistically charitable interpretation of the actions of the Ziz gang that I find it hard to understand what you really mean. If you find this at all a plausible underlying motivation for these murders I feel like you should become more acquainted with the history of violent political movements and cults, the majority of which said at some point "we're just acquiring resources that we can use for the grand revolution" and maybe even meant it.

But I don't think that contradicts the fact that she's doing what she's doing because she believes humanity is evil because everyone seems to be ok with factory farming. ("flesh-eating monsters")

Hating humans and therefore doing mean things to them is compatible with a lot of behavior, but very few of those behaviors are "taking the plight of animals seriously and fighting for them with all of your heart." Taking the plight of animals seriously and not doing obviously counterproductive or insane things in the name of "helping" them are one and the same, for me, and I don't know what else it could be.

This is a strawman.

Is it tho

1Ratios
You seem to claim that a person that works ineffectively towards a cause doesn't really believe in his cause - this is wrong. Many businesses fail in ridiculously stupid ways, doesn't mean their owners didn't really want to make a profit.
[-]lc1311

If a businessowner makes silly product decisions because of bounded rationality, then yes, it's possible they were earnestly optimizing for success the whole time and just didn't realize what the consequences of their actions would be.

If a(n otherwise intelligent) businessowner decides to shoot the clerk at the competitor taco stand across the street, then at the very least they must have valued something wayyyyy over building the business.

6localdeity
Or the businessowner's thinking process is damaged (despite being intelligent—these coexist a lot more often than we would like), and they sometimes do useful things and sometimes act counterproductively.  You could view this as "they sometimes act in a way that soothes a damaged part of their mind, which they value more highly than building the business".  Which way of viewing it is more helpful? I think the "damaged thinking" view would more likely predict that, when they're soothing the damaged part of their mind, they don't think through the consequences very thoroughly, while the other perspective—"they're 100% rational, and their values include doing some of this weird stuff"—predicts they always understand the consequences.  Now, you could add in another assumption: "They place high value on not thinking through the consequences of certain actions."  (I guess you can ultimately explain any behavior pattern by making enough assumptions about what they value.)  I don't have a strong position here on the best way of modeling it.

Not necessarily because you might also commit to stopping it in a non-escalatory way. For instance you could work to make economically viable lab-grown meat to replace animal products.

Hence the other key ingredient in Zizianism is commitment to escalating all the way, which allows things to blow up dramatically like this. (And escalating all the way has the potential to go wrong in most conflicts, not just veganism (though veganism seems like the big one here), e.g. I doubt the landlord conflict was about veganism.)

As an analogy, if you were dealing with the Holocaust, you could try to directly destroy all Nazis, or you could try to mitigate against the Holocaust in less escalatory ways (e.g. trying to have Jews emigrate from Nazi territories, which I imagine could be done either with the cooperation of Jews as in the Danish case, or with the cooperation of Nazis as in the Madagascar plan).

4Ratios
I agree with your comment. To continue the analogy, she chose the path of Simon Wiesenthal and not of Oskar Schindler, which seems more natural to me in a way when there are no other countries to escape to - when almost everyone is Nazi. (Not my views) I personally am not aligned with her values and disagree with her methods. But also begrudgingly hold some respect for her intelligence and the courage to follow her values wherever they take her.
2Nicholas / Heather Kross
(Nitpick: historians seem to generally think that the Madagascar plan wasn't even really on the table for most of the Nazi leadership.)
3tailcalled
Fair, I didn't know much about the Madagascar plan, it's just something I had heard someone bring up once.
2Richard_Kennaway
I eat meat and wear leather and wool. I do think that animals, the larger ones at least, can suffer. But I don’t much care. I don’t care about animal farming, nor the (non-human) animal suffering resulting from carnivores and parasites. I’d rather people not torture their pets, and I’d rather preserve the beauty and variety of nature, but that is the limit of my caring. If I found myself on the surface of a planet on which the evolution of life was just beginning, I would let it go ahead even though it mean all the suffering that the last billion years of this planet have seen. Bring on the death threats. Btw, I think that Zizzism was — I should say “is” now that she has been reported as still alive — about a lot more than animal welfare, although that was a part of it. But I will have to peer once again into the cesspit to confirm that.

I downvoted for disagreement but upvoted for Karma - not sure why it's being so heavily downvoted. This comment states in an honest way the preferences that most humans hold.

[-]lc4234

Well I downvoted, first because I find those preferences pretty abhorrent, and second because Richard is being absurdly confrontational ("bring on the death threats") in a way that doesn't contribute to discussion. The comment is mostly uncalled-for gloating & flag planting, as if he's trying to start a bravery debate.

Any of those things seem to me sufficient enough reasons to downvote, and altogether they made me strong downvote.

9FeepingCreature
This is just human decision theory modules doing human decision theory things. It's a way of saying "defend me or reject me; at any rate, declare your view." You say something that's at the extreme end of what you consider defensible in order to act as a Schelling point for defense: "even this is accepted for a member." In the face of comments that seem like they validate Ziz's view, if not her methods, this comment calls for an explicit rejection of not Ziz's views, but Ziz's mode of approach, by explicitly saying "I am what you hate, I am here, come at me." A community that can accept "nazis" (in the vegan sense) cannot also accept "resistance fighters" (in the vegan sense). Either the "nazi" deserves to exist or he doesn't. But to test this dichotomy, somebody has to out themselves as a "nazi."
5Said Achmiz
Yes, and also it’s a matter of maintaining the Overton window. Allowing perfectly ordinary and morally unproblematic (at worst!) things like “eating meat” and “wearing leather and wool” and “not caring about wild animal ‘suffering’” to be regarded as something one can’t admit for fear of ostracism is nothing more nor less than allowing one edge of the Overton window to move—toward Ziz. Hence: strong upvote and full agreement for Richard’s comment.
4Aiyen
How functional can our community be without pushing back against people like Ziz? Richard’s comment seems to be a way of doing so, and thus potentially useful. It’s fine if you disagree with him, but while I agree the comment was flag-planting, some degree of flag-planting is likely necessary for a healthy discussion. Consider the way well kept gardens die by pacifism (can’t link on my phone, but if you’re not familiar with it there’s an excellent Yudkowsky post of that name that seems relevant). Zizianism is something worth planting a few flags to stop.

How functional can our community be without pushing back against people like Ziz? Richard’s comment seems to be a way of doing so, and thus potentially useful.

This is basically the politician's syllogism:

  1. We must do something.
  2. This is something.
  3. Therefore, we must do this.

In general, the politician's syllogism fails because not only must we do something, but we must do something that works and doesn't cause side effects that are worse than its benefits and doesn't have too high opportunity costs etc. In this case, it's valuable for people to "push back against people like Ziz", but it's disvaluable for people to have awful values (like not caring about animal suffering despite believing it to be real), and to be hyperbolic and confrontational (as in "bring on the death threats" or describing a poorly thought-out blog as a "cesspit").

3Aiyen
Good analogy, but I think it breaks down. The politician’s syllogism, and the resulting policies, are bad because they tend to make the world worse. I would say that Richard’s comment is an improvement, even if you think it might be a suboptimal one, and that pushing back against improvements tends to result in fewer improvements. Don’t let the perfect be the enemy of the good is a saying for very good reason. The syllogism here is more like: 1. Something beneficial ought to be done 2. This is beneficial. 3. Therefore I probably ought not to oppose this, though if I see a better option I’ll do that instead of doubling down on this.
4DanielFilan
It could be that Richard's comment is actually good. I still think that the argument I quoted fails to establish that, for the same reason the politician's syllogism doesn't work.
2Richard_Kennaway
Given Ziz's explicit calling for people to die, I don't think there is anything hyperbolic about my "bring on the death threats". Ziz's blog is not "poorly thought-out", it is a condensed nugget of evil. I am not the only one here to have observed this. So here we are.
1lc
LessWrong is not her blog.
2Richard_Kennaway
Of course it isn't. Her blog is sinceriously.fyi and that is what I was referring to.
0lc
So go tell her, is my point.
4Richard_Kennaway
I understand from what was posted here that she is currently, or at least recently, in police custody under suspicion of murder. [ETA: Correction: in custody for obstructing police investigation; separately, under suspicion of murder.] Anyway, I'm addressing the LW audience, not Ziz. You know, the people who are disagreeing with what I said but (according to the karma) not on average disagreeing with my having said it.
-6Richard_Kennaway
7npostavs
I haven't voted at all, but perhaps the downvotes are because it seems like a non sequitur? That is, I don't understand why Richard_Kennaway is declaring his preferences about this.
4Richard_Kennaway
Because there is a great want of people saying this. Someone must plant this flag, given the forest of flags already waving on the side of the enormous suffering of everything from cows to cockroaches to bacteria to Bing. But it doesn't feel like waving a flag when everyone around you is waving the same one. It feels like the flags are just reality itself. I notice that while there are a lot who disagree with my post (manifested by the agreement-downvotes), its karma has bobbed up and down since I posted it and currently stands positive.

Actually, this makes me think of something.

We sometimes see with rationalists and utilitarian EAs do something like the same thing we worry about with AI: unaligned optimization that produces outcomes we don't like. Unfortunately, because humans disagree on norms/ethics/values, it's kind of hard to know the difference between "going off the rails" and "correcting a massive oversight or collective moral failing", especially from the inside.

I'm gonna add an even more pessimistic hypothesis: That the disagreements around values are fundamentally irresolvable because there is no truth at the end of the tunnel.

Or, one man's "going off the rails" is another man's "correcting a massive oversight or collective moral failing", and these perspectives can't be reconciled.

Also related: Scott Alexander on epistemic learned helplessness: https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/

7Raemon
Not quite the main point here, but, my brain has always failed to really cache "Reason as a memetic immune disorder" as a useful concept. When I look at the Memetic Immune System tag, I'm not quite sure how all the posts are supposed to relate to each other. I'd be interested in someone writing up a tag description that actually uses "memetic immune system" in a sentence and gives some examples of how the posts relate.

In my head this is related to Scott's Robustness to Relative Scale. Reasoning is great, but if you take a particular agent and scale up the part of it that does explicit reasoning by 100x whilst leaving the rest at the same power level, then it may overpower other parts of the system designed to keep it in-check. 

To tell an overly specific concrete story of how this might happen:

"I have some self-deception processes that inspire my reasoning, and I have some ethical conscience part that also inspires my reasoning. My reasoning is now good enough to always beat the ethical conscience arguments if it wants to, so when I'm motivated to do that cognition, I always win. My reasoning got better, and now when my selfishness and my conscience butt heads, my selfishness always wins the argument."

1Noosphere89
One of the bigger issues is that when we make moral judgements on others, we probably don't realize that we are imposing our own values, and that this is all we can do. This cashes out in 2 pessimistic claims: 1. Fundamentally unbridgeable divides between values exist, as well as their intensity, and this is poor from the perspective of compromise. 2. Rationality increases don't have anything to do with terminal goals changes.

Faith is the defence of reason against the passions.

The memetic immune system is the defence of the passions against reason, for while True Reason is eternally perfect, it is not so when attempted by our fallible minds.

We can be as mistaken in our reasoning as in our passions, and both must work harmoniously together.

I'd rather see a shorter, commoner single word for "memetic immune system". WWCW? (What Would Chesterton Write? Or C.S. Lewis, or Aquinas in English translation.)

2Viliam
Adding Up to Normality? Common sense? Ignoring base rates? -- in the sense that if you just invented a clever theory why good is actually bad, and bad is actually good, consider that the prior probability of this being true (and you being the first person who noticed that) is smaller than the probability of you making a mistake in the clever argument. Maybe even tails coming apart -- in the sense that arguments that seem rational are in general more likely to be true, however the correlation may disappear at the extremes, and the marginal value of taking one more idea in an already too long unlikely chain seriously may be negative.
1Aiyen
Well said! Though it raises a question: how can we tell when such defenses are serving truth vs defending an error? As for an easier word for “memetic immune system”, Lewis might well have called it Convention, as convention is when we disregard memes outside our normal mileu. Can’t say for Chesterton or Aquinas; I’m fairly familiar with Lewis, but much less so with the others apart from some of their memes like Chesterton’s Fence.
4localdeity
With a couple of minutes' thought: A memetic immune system is whatever allows or rejects new memes as one encounters them, and kills previously accepted memes.  Reason is a piece of it that, among other uses, kills falsehoods.  That means it kills protective and comforting lies, approximations to the truth, etc., which might have had their purposes in the past.  If, for example, your reason is strong enough to kill "I should behave well because God will punish me if I don't", but not strong enough to come up with a good moral framework to replace it, then you may have a problem while that situation persists.
2Noosphere89
Re reason: The real issue is that whenever we make moral judgements, we probably don't realize that we are imposing our values on it. That's why from the perspective of a fanatic like Ziz doesn't believe they're irrational, primarily because the Zizians and LW have fundamentally unbridgeably opposed values.
6maia
I don't think "taking ideas too seriously" is what went wrong here. Their actions are just too insane and frankly random and nonsensical to fit that model.

“Taking ideas too seriously” is unlikely to be what went wrong with Ziz, but I do think that it’s a large part of what went wrong with all the misguided individuals who’ve allowed themselves to be drawn into Ziz’s orbit, have adopted Ziz’s bizarre perspectives, etc.

2Raemon
My guess is that "taking ideas seriously" played a role in the chain somewhere, but some of the ideas are about how to think-internally, or the second-order effects of having taken something seriously.

Who is Ziz and what relation does she have to the rationalist community?

Ziz is a blogger at sinceriously.fyi. She used to hang around the rationality community a fair bit (I met her at a CFAR/MIRI workshop). She has since decided that some figures and institutions in the rationality community are worth protesting (see e.g. this protest at a CFAR reunion, "Jack LaSota" is Ziz). More, possibly biased information is available at zizians.info.

[-]lsusr1118

Ziz's blog is openly, obviously evil. And not in a fun, trolly way—or even inadvertent mundane evil. Boringly explicit evil, with literal endorsement of the Star Wars Sith religion.

7[anonymous]
Would it be evil from the perspective of someone with priors who believes animal lives are equal in value or greater than humans lives? Presumably Zizians don't believe they are evil.

Here's a quote from Ziz's post My Journey to the Dark Side.

Reject morality. Never do the right thing because it’s the right thing. Never even think that concept or ask that question unless it’s to model what others will think. And then, always in quotes. Always in quotes and treated as radioactive. Make the source of sentiment inside you that made you learn to care about what was the right thing express itself some other way.

Here's a quote from Ziz's post Neutral and Evil.

If you’re reading this and this is you, I recommend aiming for lawful evil. Keep a strong focus on still being able to coordinate even though you know that’s what you’re doing.

An evil person is typically just a neutral person who has become better at optimizing, more like an unfriendly AI, in that they no longer have to believe their own propaganda. That can be either because they’re consciously lying, really good at speaking in multiple levels with plausible deniability and don’t need to fool anyone anymore, or because their puppetmasters have grown smart enough to be able to reap benefits from defection without getting coordinated against without the conscious mind’s help.

-1[anonymous]
So this morality would be one of "optimized selfishness"? "Do whatever is best for me, knowing that means it will be on balance evil"? The morality of Bernie Madoff? Interesting.
[-]Raemon6550

FYI I don't think this is the right summary. Ziz's morality is something more like "society's conception of good is corrupt, therefore you should be prioritizing unlocking yourself from society's frame". 

They have a bunch of complicated worldview relating to how to do this. I do think they go off the rails, and given how many people ended up either committing suicide or getting involved in ways that seemed to make their life worse, I do not recommend trying to follow their worldview and understand the details of it. 

But I wanted to flag this because, if you start with an assumption "Ziz is cartoonishly Star Wars Evil", and then you start reading any of their content, you might notice "oh wait this isn't as cartoonishly Star Wars Evil as it sounded at first glance, maybe there's something to this." And then you might frogboil yourself into taking it too seriously.

(I might have slightly more nuanced advice on how to relate to Ziz for people I'm having an in-depth 1-1 conversation with, but, this is the default advice I feel good about sharing in a low-fidelity way)

3[anonymous]
Thanks.  I'm imagining a strident infohazard sign warning not to even think about Zizianism.   What is interesting is more than such infohazards are possible.  If you think about it, the cheapness of modern communications and rapidly improving AI may allow for the creation of infohazard weapons - media content that often kills humans. But yeah, thanks for the warning.  "Zizianism" is some dangerous stuff I and other mere humans should avoid studying closely.  Leave that information in a sealed box.

To offer a contrasting viewpoint:

I’ve read some of the stuff on Ziz’s website. In my experience, there were a few scattered bits here and there that were sensible (those were the things that were basically restatements of views found in plenty of other places, e.g. most of this post—until the last handful of paragraphs, where it goes off the rails—is an insightful analysis of one of the fundamental problems related to NVC and similar techniques… but of course plenty of other people have written about this sort of thing).

The rest was just very obviously wrong and insane. I found myself utterly baffled by the notion that anyone could even be tempted to take any of it seriously, or believe it, etc. My reaction wasn’t “oh no, this sounds disturbingly plausible!”; it was “wow, this is sheer nonsense—the deranged ramblings of a very obviously mentally disturbed individual”.

Now, not everyone reacts to this stuff like I did—obviously! But the right takeaway, I think, isn’t “this ‘Zizianism’ is dangerous, mere humans shouldn’t consider it too closely”. Rather, the takeaway is a question: “what mental quirks make some people incapable of seeing this for the insane absurdity that it is?” Why do some people find this stuff plausible? And: by what means can we identify such tendencies in ourselves, and counteract them?

[-]lc439

Ziz's tendency towards inscrutable metaphors and loaded jargon serve essentially the same purpose as typos in a spam email - it's meant to filter you out. If the hypnotic language made more literal sense, it would bring in people who weren't especially susceptible in particular to Ziz's brand of rhetoric, and thus might not be good recruits for her criminal organization.

it captures the sort of person who gets hooked on tvtropes and who first read LW by chasing hyperlink chains through the sequences at random. It comes off as wrong but in a way that seems somehow intentional, like there's a thread of something that somehow makes sense of it, that makes the seemingly wrong parts all make sense, it's just too cohesive but not cohesive enough otherwise, and then you go chasing all those hyperlinks over bolded words through endless glossary pages and anecdotes down this rabbit hole in an attempt to learn the hidden secrets of the multiverse and before you know what's happened it's come to dominate all of your thinking. And there is a lot of good content that is helpful mixed in with the bad content that's harmful, which makes it all the harder to tell which is which.

the other thing that enabled it to get to me was that it was linked to me by someone inside the community who i trusted and who told me it was good content, so i kept trying to take it seriously even though my initial reaction to it was knee-jerk horror. Then later on others kept telling me it was important and that i needed to take it seriously so i kept pushing myself to engage with it until i started compulsively spiraling on it.

8Said Achmiz
Hmm, no, I don’t think so. I first read LW (well, it was OB at the time, but same deal) by chasing hyperlink chains through (what would come to be called) the Sequences at random. And I’ve read my share of TV Tropes. So this doesn’t check out. Whatever the culprit quirk is, it’s clearly got nothing to do with whatever it is that makes people… read things by clicking on hyperlinks from other things. Hmm, I see. Would you say that the problem here was something like… too little confidence in your own intuition / too much willingness to trust other people’s assessment? Or something else? (Did you eventually conclude that the person who recommended Ziz’s writings to you was… wrong? Crazy? Careless about what sorts of things to endorse? Something else?)

Hmm, I see. Would you say that the problem here was something like… too little confidence in your own intuition / too much willingness to trust other people’s assessment? Or something else?

that was definitely a large part of it, i let people sort of 'epistemically bully' me for a long time out of the belief that it was the virtuous and rationally correct thing to do. The first person who linked me sinceriously retracted her endorsements of it pretty quickly, but i had already sort of gotten hooked on the content at that point and had no one to actually help steer me out of it so i kept passively flirting with it over time. That was an exploitable hole, and someone eventually found it and exploited me using it for a while in a way that kept me further hooked into the content through this compulsive fear that ziz was wrong but also correct and going to win and that was bad so she had to be stopped.

Did you eventually conclude that the person who recommended Ziz’s writings to you was… wrong? Crazy? Careless about what sorts of things to endorse? Something else?

The person who kept me hooked on her writing for years was in a constant paranoia spiral about AI doom and was engaging with Zi... (read more)

7Said Achmiz
I see, thank you.
5Richard_Kennaway
Insufficient defence of the passions against reason, then?
1Slimepriestess
something like that. maybe it'd be worth adding that the LW corpus/HPMOR sort of primes you for this kind of mistake by attempting to align reason and passion as closely as possible, thus making 'reasoning passionately' an exploitable backdoor.
[-]Viliam2413

I believe that reading about Zizianism is not dangerous. Actually meeting Ziz and debating them for a long time is. (Reading is only dangerous indirectly, as it may make you curious.) Kinda like a difference between reading a Scientology book, and joining an actual Scientology organization.

One of the tricks Ziz uses is redefining the meaning of the words (including words such as "good" and "evil", or even "person"). This works much better if you are overwhelmed, and do not have enough time to track the relations of Zizian jargon with actual words. The trick works -- and this is what many cults do -- by attaching your cached connotations of the old words to the new ones.

*

As an example, imagine the word "good". If you are like me, you probably do not have an exact definition, but you still have a vague idea that "good" is somehow correlated to helping people and anticorrelated to hurting them. And you probably have a cached thought like "I want to be good (perhaps unless the cost is too high)".

Now imagine that Ziz gives you a very complicated argument why "good" should be redefined to... something very abstract and complicated, based on many incorrect assumptions... but in effect, no... (read more)

7Richard_Kennaway
Unless you've studied until the screaming comes from within you. In the present context, try the "Morality" section. (For reasons, I designed that page so that the subsections cannot be directly linked to.) And then the "Pure Insanity" section. I didn't dream up the contents of that page. I just took ideas that are in the air of the LW/EA/rationalsphere (and some other places, but mostly from there), and simulated "taking ideas seriously" turned up to eleven. It's intended as a vaccine, not a pathogen, but anyone who may be susceptible to taking ideas seriously might be wise to avoid looking.
6Viliam
True. Still it seems to me (maybe I am wrong here) that Ziz actually had to use the sleep deprivation et cetera in order to convince most people to buy the "up to eleven" version. Even people who take ideas more seriously than usual, often seek some kind of social approval before jumping off the deep end.
3TekhneMakre
This is very important and subtle. A real leader absolutely must understand that there is such a thing as lack of common knowledge. Anyone who is acting as though the lack of common knowledge is just you being disloyal / intentionally dumb / etc., is trying to be a cult leader.
1[anonymous]
Thank you for this post, and it suggests how we could know when an AI system is capable of similar manipulation, and when it can't. For example, a system that is accessed via a browser tab that forgets everything after a token limit is obviously not capable of this. However, a system connected to always on home devices or is given privileged access to your OS desktop (for example if 'cortana' can render itself as a ghostly human female that is always on top in windows, and is able to manipulate information in microsoft office applications for you) - and the machine has long term state so it can track it's brainwashing plans, it would be possible. (it seems obvious that at a certain point, scammers and other hostile organizations would adopt AI for this purpose)

There's this guy Michael Vassar who strikes me - from afar - as a failed cult leader, and Ziz as a disciple of his who took some followers in a different direction. Even before this new information, I thought her faith sounded like a breakaway sect of the Church of Asmodeus.

Michael Vassar was one of the inspirations for Eliezer's Professor Quirrell, but otherwise seems to have little influence.

At the risk of this looking too much like me fighting a strawman...

Cults may have a tendency to interact and pick up adaptations from each other, but it seems wrong to operate on the assumption that they're all derivatives of one ancestral "proto-cult" or whatever. Cult leaders are not literal vampires, where you only become a cult leader by getting bit by a previous cult leader or whatever.

It's a cultural attractor, and a cult is a social technology simple enough that it can be spontaneously re-derived. But cults can sometimes pick up or swap beliefs & virulence factors with each other, when they interact. And I do think Ziz picked up a few beliefs from the Vassarite cluster.

I can dig up cases in Ziz's writing where Ziz has interacted with Vassar before, or may have indirectly learned things from him through Alice.

Doesn't make Vassar directly responsible for Ziz's actions. I think Vassar is not directly responsible for Ziz.

I do want to spell this out, because I'm reading a subtle implication here, that I want to push back against.

[-]Viliam1910

For example, many abusive partners reinvent half of the cult techniques on their own. If you are the right kind of personality, it is probably enough to carefully observe your victim, and gradually remove everything that empowers them. (Their parents disapprove of you? No more contact with the parents. An article about abusive relations that actually explains a lot? No more reading that specific website or a book. Ideas encountered in free time that you don't like? Invent lots of busywork, no more free time. Thinking too much? Create emotional drama, so that the victim thinks about made up problems instead.)

Cults are simply groups that stumbled upon the right combination of techniques, efficient enough to either keep their members trapped reliably, or to recruit new members faster than the old ones leave. It helps if you can copy some techniques from your previous cult, but it is not necessary. You can also copy from abusive parents, or if you are an abusive person yourself (and Ziz seems to be) simply learn from your previous experience.

7TekhneMakre
Great comment. Especially this. A refinement: create an atmosphere of "honesty" (which caches out as perverse openness for thee, but not for me). Use your perverse openness to track when you are going down paths that might end in you breaking away. Selectively create drama and ultimatums and loyalty tests in those cases, which is more precisely targeted than just thinking too much. (This may sound crazy but it actually happens. See e.g. NXIVM; if you don't tell all your thoughts to Keith Raniere, you're resisting / in ethical breach / etc.)

I heard that LaSota ('ziz') and Michael interacted but I am sort of under the impression she was always kind of violent and bizarre before that, so I'm not putting much of this bizarreness down to Michael. Certainly interest in evidence about this (here or in DM).

It sure sounds like you think outsiders would typically have the "common sense" to avoid Ziz. What do you think such an outsider would make of this comment?

6tailcalled
Scott Alexander seems to have withdrawn some of his critiques of Michael Vassar.

I recognize that there are privacy considerations that (probably) prevent it, but I would like to see more of a public accounting of the evidence that caused Scott Alexander to withdraw those critiques, along with what critiques remain, and what he thinks caused his & the community's initial view.

My impression was that Scott previously thought the hypothesis 'interacting with Michael Vassar can cause psychosis' was worth taking seriously (in this comment), but then later decided this was mostly correlation not causation and apologized to both Michael and Jessica (in this comment).

Insofar as this is a case of the community reflexively scapegoating, it seems like we ought to examine it more carefully; insofar as there are still possible cult-like/bizarre negative effects, it seems like we should investigate those too.

4Ben Pace
I think mostly somewhat confused?  Though I've never met her, from her writing and things others have told me, I expect LaSota seems much more visibly out-of-it and threatening than e.g. Michael does, who I have met and didn't seem socially alarming or unpredictable in the way where you might be scared of a sudden physical altercation.
[-]Buck1510

I think Vassar is alarming and unpredictable in a way that causes people to be afraid of a sudden physical altercation. For example, I have felt scared of physical altercations with him. If I recall correctly, he raised his voice while telling a friend of mine that he thought they were worse than the Nazis during a conversation in a hotel lobby, which freaked out other people who were in the lobby (I don't remember how my friend felt).

2Ben Pace
Oops, I was unclear in my last line. I was attempting to distinguish between someone getting angry with you and shouting at you and then punching you, and someone who is quiet and doesn’t look at you and isn’t talking much to you who then walks over and punches you. Both are alarming. To me the latter feels more unpredictable and more alarming because I’m getting no info about when it will happen, but if someone is getting visibly angry and escalates and you don’t know where their lines are for conflict, then I can see that being more intensely alarming.
8DanielFilan
For what it's worth, when I've talked to Ziz, she didn't seem out of it, threatening, alarming, or unpredictable to me, and I wasn't scared of a sudden physical altercation.
7Gordon Seidoh Worley
Interesting. I remember meeting Ziz once before they were fully out as Ziz, and it was clear to me that their vibe was "off" but given so many rationalists have weird vibes it was hard to make much of. It's easy with hindsight bias to try to give myself points for noticing what might happen, but actually that was pretty unclear to me right up until the CFAR protest happened and then lots of details about what had been doing on came out. What I mean by vibe being off: many rationalists don't behave in normal ways and don't play into standard social scripts, so they fall into this weird space where it can be hard to predict what they will do and say in the next few moments so it can create something like an uncanny valley of social interaction. But I get this from lots of people who haven't been accused of homicide/accessory to homicide, so given the base rates it's hard to make strong predictions here.
5Raemon
I think Ziz's early vibe was "off/weird, but not in a particularly unusual-for-rationalists-way." I definitely see the seeds of their worldview in their early writing/talking/in-person-interactions that make me not super surprised at how things turned out. But, I do think it's important that the vibe didn't feel threatening/alarming in a physical way.
7Slimepriestess
There was also definitely just an escalation over time. If you view her content chronologically it starts as out as fairly standard and decently insightful LW essay fair and then just gets more and more hostile and escalatory as time passes. She goes from liking Scott to calling him evil, she goes from advocating for generally rejecting morality in order to free up your agency to practicing timeless-decision-theoretic-blackmail-absolute-morality. As people responded to her hostility with hostility she escalated further and further out of what seemed to be a calculated moral obligation to retaliate and her whole group has just spiraled on their sense that the world was trying to timelessly-soul-murder them.
4DanielFilan
I could well just be bad at reading vibes.

I think it's good that this post was written, shared to LessWrong, and got a bunch of karma. And (though I haven't fully re-read it) it seems like the author was careful to distinguish observation from inference and to include details in defense of Ziz when relevant. I appreciate that.

I don't think it's a good fit for the 2023 review. Unless Ziz gets back in the news, there's not much reason for someone in 2025 or later to be reading this.

If I was going to recommend it, I think the reason would be some combination of

  • This is a good example of investigative journalism, and valuable to read as such.
  • It's a good case study of a certain type of person that it's important to remember exists.

But I don't think it stands out as a case study (it's not trying to answer questions like "how did this person become Ziz"), and I weakly guess it doesn't stand out as investigative journalism either. E.g. when I'm thinking on these axes, TracingWoodgrains on David Gerard feels like the kind of thing I'd recommend above this.

Which, to be clear, not a slight on this post! I think it does what it wanted to do very well, and what it wants to do is valuable, it's just not a kind of thing that I think the 2023 review is looking to reward.

Most LessWrong readers do not attend meetups, and this is basically useless to them. Some readers do attend meetups, which Ziz will not attend because the organizers are aware of this and are will keep Ziz out. Some organizers aren't aware, and this is a useful thing to be able to point to in that case, though since this was written describing a developing situation it would be kind of nice to have a conclusion or update somewhere near the top.

Overall, I wouldn't want this in the Best Of collection, but I do expect to link people to it in the future.

What would happen if I got some friends together and we all decided to be really dedicatedly rational?

This is an important scenario to reason about if I want to be a rationalist, and I think my predictions about that scenario are more calibrated than they would be in a world where I didn't read this post. Specifically, my predictions in light of this post have way, way fatter tails.

4Eli Tyre
If your takeaway is only that you should have fatter tails on the outcomes of an aspiring rationality community, then I don't object. If "I got some friends together and we all decided to be really dedicatedly rational" is intended as a description of Ziz and co, I think it is a at least missing many crucial elements, and generally not a very good characterization.   
1Hastings
It is intended as a description of Ziz and co, but with a couple caveats:  1) It was meant as a description that I could hypothetically pattern match to while getting sucked in to one of these, which meant no negative value judgements in the conditions, only in the observed outcomes. 2) It was meant to cast a wide net - hence the tails. When checking if my own activities could be spiraling into yet another rationalist cult, false positives of the form "2% yes- let's look into that" are very cheap. It wasn't meant as a way for me to police the activities of others since that's a setting where false positives are expensive.