A big problem I see is that if the panel explains their reasoning, it exposes them to a libel lawsuit, but if the panel does not expose their reasoning, it allows the bad actor to publish their own bullshit version unopposed.
(Specifically, I have seen "they simply don't like me because I am politically incorrect" written by people who were actually banned for doing specific bad things to other people in the rationalist community; things that were completely unrelated to politics or anything in that style.)
Bingo, the root problem is pretending to have any quasi-judicial structure/authority at all.
People of roughly equal status issuing 'judgements' or 'decisions' on each other really doesn't make sense for that reason, at best you can do so within a private club and its property lines.
A federation of private clubs may decide to do so, very rarely, only for the most serious cases, because as mentioned in the OP there's always the risk of some clubs siding with the accused and then deciding to leave, splitting the federation.
I've also learned: It is easier to remove good people from your community than to remove people who are aggressive, manipulative, narcissistic, or psychopathic, because the former will remain reasonable throughout the process, while the latter will fight dirty.
To give a concrete example: the original ACDC memo says Brent cooperated with the panel while the complainant did not, and that's one reason they trust him (recalled from memory, didn't double check my wording). Which is insane; an abuser who is good at manipulating people and not traumatized finds it easier to engage in apparent good faith with an investigating org than a traumatized victim. You can try and fix this by being overtly sympathetic towards the victim, but that's either biasing the outcome or a lie.
I feel like this is the opposite of the quoted text? Or your example is of the bad actor both "remaining reasonable" and "fighting dirty"
Fighting dirty can involve looking reasonable to the outside, e.g. being willing to lie or bend the truth and distracting from the key issues of the matter - these can all be done civilly.
But while I can certainly see failing "to engage in apparent good faith" is like an understandable reaction, it doesn't really feel like a central example of "remain[ing] reasonable throughout the process"
They reasonably chose to disengage with the process that was miserable for them and had a low expected pay off.
I've seen a bit of this in some organizations I've been part of. The most important part I see missing is enforcement powers. If you have a group of excellent and sage judges who can impartially consider the facts but all they can do is issue advisory opinions, all you have is another social bloc taking one side or the other in an interpersonal debate. You have gossip and the whisper network cosplaying a court of law. You have nothing.
I have not the first clue how to handle this outside of a formal organization, but solving this in an organization with a real structure is at least a step forward. Not naming names because I don't speak for them, but one organization I've seen do this well had a small elected committee that handled any complaints about members. They did basic investigation, worked with both accuser and accused, and normally focused on repairing the harm and reintegrating both people into the community (the keyword here is restorative justice, if you want details). Often this caught troublemakers early, and people have been accused of wrongdoing, gone through this process, and remained members of the organization with much drama. This works because the committee also had a hammer: they could ban people from attending organization events temporarily while they put permanently removing the person from the organization to an organization wide vote. While I was part of the organization, no one was removed by a vote, but people have been removed by vote before, and people have quit the org when it became clear that there would be consequences for the bad actions due to this process.
Importantly, the committee was explicitly held to far lower standards than a court of law. They were expected to be impartial (and not handle a case if they couldn't be), but no one was spending hundreds of hours on a single case. There are no lawyers, no rules of evidence. The trade-off for the whole process being far less formal and less work - not just for the committee members but also for the accuser and accused - is the relatively lightness of the penalties. Defendants in a criminal court need the full protection of formal systems of law because the court can take away your money, your freedom, or even your life. The committee I'm talking about can at worst take away your membership in one organization, and even that has a safeguard. The people who wrote the rules for the committee understood that not being a massive burden on everyone involved was worth the trade-off of less protections. The focus on repairing and preventing harm over punishment also helped - the consequences of ruling the wrong way on something are minimized here, which helps smooth over any wrong decisions.
It's also worth making it explicit that when this sort of issue comes up, of how to handle accusations of wrongdoing in organizations or groups without involving the courts, it's almost always about rape. The whisper networks or public accusations that these sorts of more formal structures try to avoid aren't about someone getting beat up or stolen from, they're almost always about rape, other sexual assault, or behavior that make people feel like they're in danger of being raped. If someone beats you up or steals from you, you don't have to start a whisper campaign, you go to the police (or a lawyer in some cases). Explicitly saying something I don't have proof of, epistemic status "confident but maybe you shouldn't be": this is because the (American) courts are bad at dealing with rape and sexual assault, and in the absence of formal systems of justice you get informal systems of justice, and those tend to suck for everyone, accuser included (public accusations and whisper campaigns aren't a great way to achieve justice, but if they're what you have, they're what you have).
I agree with most of this, except for the part "if someone steals from you, you can go to police". Yes, if someone literally grabs your purse and starts running away, call the police.
But there are also situations that are less clear, for example when people didn't write a contract because they trusted a person from the same community, when they just made a verbal agreement and when the time came for the other person to do what they said, they denied it or said it was all a misunderstanding. Sometimes you have no legal evidence.
Imagine you and me renting a car together for a trip. You rent the car and pay for the gas, and the agreement is that when the trip is over, I will pay you 50% of the expenses. Except that afterwards I don't give you a cent, I claim that this was a misunderstanding, that I sincerely believed that you were offering me a free ride from the goodness of your heart otherwise I would never have agreed to that, and I am a poor person and don't have so much money anyway. (For bonus points, I can act offended and accuse you of playing dirty tricks on me.) Now what? It is your name on the car rent, you paid cash for the gas, I deny everything. I don't think you would have a good case in court, and even if you did, it is probably not worth the effort. You will remember never to trust me again, but next time I will simply misuse someone else.
Situations like this are typically addressed by gossip, but it is your word against my word, and maybe I am more popular than you because I am a charismatic sociopath. Or I may threaten to sue you for slander. Or I may proactively share my story first (how you tried to scam me out of money by offering me a free ride and then trying to make me pay you an absurd amount of money -- for better effect I will exaggerate the number) and warn people against trusting you. Or I may start an unrelated conflict with you, for example a political debate on LW or Facebook, and then insist that you just made up the story about the car as a punishment for me expressing insufficiently woke opinions. Etc.
Now what a panel can do in such situation is to listen to both sides, make no conclusion at first, but if I keep having "misunderstandings" with more people who have no past record of similar behavior, after two or three it cumulatively becomes a strong Bayesian evidence that I am actually the bad guy. Even if there is still zero evidence admissible in court.
"if I keep having "misunderstandings" with more people who have no past record of similar behavior, after two or three it cumulatively becomes a strong Bayesian evidence that I am actually the bad guy." It's not quite that easy. Abuser's may particularly tend to seek out vulnerable people, and it's a real effect that when you are already raising a complaint about someone, this may open you up to further abuse by other bad actors, who can now have exploit that you now have spent your social capital. In other words, Bayesian considerations have a place, but you need to be extra careful that you're not misattributing the correlations.
Yes, that makes sense. But if we already assume that the truth may be impossible to figure out reliably, no solution is going to be perfect.
(There is also a similar problem, that some people are more likely to be falsely accused than others.)
You're correct. I wish we had any sort of tradition that let people with a minor dispute go before some neutral party without expense or bureaucracy - less in the sense of court of extremely small claims, and more that people should be more willing to say to a trusted friend, "hey, resolve this dispute for us and we'll buy you dinner." Then again, this requires you to both trust the same person, and for neither person to be acting in such bad faith that they refuse the process. If there's a default place people can go, with very low costs in time and no cost in money, refusing to even have an argument heard would be a pretty big strike against you in most situations, and as you said if the panel has multiple cases/ complaints against a person that's at least Bayesian evidence that they're doing something wrong.
"It's true that we don't want women to be driven off by a bunch of awkward men asking them out, but if we make everyone read a document that says 'Don't ask a woman out the first time you meet her', then we'll immediately give the impression that we have a problem with men awkwardly asking women out too much — which will put women off anyway."
This seems a weak response to me, at best only defensible considering yourself to be on the margin and without thought for longterm growth and your ability to clarify intentions (you have more than 3 words when interacting with people irl).
To be clear explicitly writing "don't ask women out the first time you meet her" would be terrible writing and if that's the best writing members of that group can do for guidelines, then maybe nothing is better than that. Still, it reeks of "we've tried for 30 seconds and are all out of ideas" energy.
A guidelines document can give high level guidance to the vibe you want (eg. truth seeking, not too much aggressiveness, giving space when people feel uncomfortable, communicating around norms explicitly), all in the positive way (eg. you say what you want, not what you don't want), and can refer to sub-documents to give examples and be quite concrete if you have socially impaired people around that need to learn this explicitly.
I think another big issue with codes of conduct is that they just shift the burden around. You're still left with the issue of interpreting the spirit of the norm, deciding if everyone at least made a good faith attempt to stick to it, if good faith is enough, etc. I don't have much experience with them but I honestly don't know if they help that much. Seems to me like there are two types of "troublemakers" in communities no matter what:
- people who are purposefully deceptive and manipulative;
- people who simply lack the social grace and ability to "read the room" required to meet other's expectations of social norms adherence rather than just stick to their own interpretation of them.
Type 1 you want to kick out. Type 2 you ideally want to be a lot more graceful and forgiving with, though in some extreme cases you might still need to kick them out if their problems are unfixable and they make no effort whatsoever to at least mitigate the issues. Writing the rules down doesn't help as long as they're flexible, because the problem those people have is a lack of the sort of intuition that others possess for grokking flexible rules altogether. And if you make them inflexible you just have a chilling effect on every interaction, and throw away a lot of good with the bad. After all, for example, why shouldn't someone ask a woman out at their first meeting if they're both clearly into each other and sparks are flying? These things happen! And people should be able to give it a try, I think it's important to make it clear that there's nothing sinful or bad about courtship or flirting per se; too many rigid rules about such personal interactions inevitably carry a sort of puritanical vibe with them, regardless of intention. But as usual, "use your best judgement" has very uneven effects because some people's best judgement is just not that great to begin with, often through no fault of their own.
It would be better if the "guardians of proper behavior" had options beyond "permaban or nothing".
I mean, for serious wrongdoing -- permaban; of course. But for mere annoying behavior -- verbal admonishment in private, verbal admonishment in public, second admonishment, temporary ban, and only after everything else fails, permaban. This would be a more gentle approach.
Counter-intuitively, the gentler approach would require more authority and trust in the guardians. The verbal admonishment needs to be taken seriously by everyone else, otherwise the admonished person will not take it seriously, and thus it is a waste of time. (Unfortunately, as I know my people, the most likely reaction would be a loud and long debate about what are the proper norms, what is the best algorithm to decide what are the norms, whether the guardians are overstepping the line, and whether telling someone "dude, she said she was not interested, leave her alone" means that the entire community is epistemically rotten, corrupted by wokeness, and tomorrow the wrongthinkers will be probably be taken to death camps.) It is important for the admonishments to be cheap (cognitively and emotionally) for the guardians, otherwise they will hesitate to react, and the entire system will revert back to "we only act when the situation is clearly horrible", in which case the appropriate outcome is indeed the permaban.
Less Wrong has put a lot of effort into creating other options and automating them. Downvoted allow subtle user feedback. Mass sock puppet voting previously ruined down votes, so they put effort into controlling sock puppets. Weighted votes so trusted people have more impact. Too many downvoted comments will lead to throttling but not bans.
My sense is these have improved things a lot, but we're also far from my ideal outcome.
"It's true that we don't want women to be driven off by a bunch of awkward men asking them out, but if we make everyone read a document that says 'Don't ask a woman out the first time you meet her', then we'll immediately give the impression that we have a problem with men awkwardly asking women out too much — which will put women off anyway."
American social norms around romance continue to be weird to me. For the record, y'all can feel free to ask me out the first time you meet me, even if you do it awkwardly ;)
I think this might be typical-minding. The consequences of this dynamic are actually pretty serious at macro-scale e.g. due to reputation of meetups, and evaporative cooling of women and high-status men as they avoid public meetups and stop meeting people new to AI safety.
I'm glad to hear there's people who don't let it get to them, because it is frankly pretty stupid that this has the consequences that it does at the macro-scale. But it's still well-worthy of some kind of solution that benefits everyone.
I honestly don't know. The discussions of this problem I encountered are all in the American (or at least Western) context[1], and I'm not sure whether it's because Americans are better at noticing this problem and fixing it, or because American men generate more unwanted advances, or because American women are more sensitive to such advances, or because this is an overreaction to a problem that's much more mild than it's portrayed.
Also, high-status men, really? Men avoiding meetups because they get too many propositions from women is a thing?
- ^
To be clear, we certainly have rules against sexual harassment here in Israel, but that's very different from "don't ask a woman out the first time you meet her".
I think the implication was that "high-status men" wouldn't want to hang out with "low-status men" who awkwardly ask out women
1I used the word "high-status men" as a euphemism that I'm not really comfortable talking about in public, did not notice it would be even harder to get for non-americans. My apologies.
I used "high-status men" mainly as the opposite of low-status men, in that they are men who are low status due to being short, ugly, unintelligent, or socially awkward, sufficiently so that they were not able to gain social status. These people are repellent to other men as well as women, sadly. @Roko has been tweeting about fixes to this problem such as reforms in the plastic surgery industry, and EA and rationalists are well above base rate communities (e.g. classical music society) for tolerating/improving low social skills and male shortness. This is due to primate instincts which usually cannot be overcome, in spite of people feeling optimistic about their ability to overcome them. The degree of social awkwardness is defined/measured by the harm it does someone; if someone looks "socially awkward" but in a charming or likable way that remains charming or likable, that is not a serious (or even significant) case, as it does not doom someone to low social status.
This is also a reason why so many people have so little tolerance for non-transhumanists as a class of ideologues; non-transhumanists accept the status quo of our current tech level, where human genetic diversity dooms a large portion of people to a pointlessly sad and miserable life without their consent, (on top of dooming everyone to a short life).
When people found out about ACDC's previous ruling on Brent, many were appalled that ACDC had seen the evidence laid out in the Medium posts and ruled that it was okay for Brent to continue on like that
As I recall, the ACDC had in fact not seen the evidence laid out in the Medium posts. (One of the panelists sent an email saying that they had, but this turned out to be incorrect--there was new information, just not in the section he had read when he sent the email, and prematurely sending that email was viewed as one of the ACDC's big mistakes, in addition to their earlier ruling.)
When joining a workplace, a university, or a club sport, you are opting into a code of conduct, including the part where there's an entity that enforces that code of conduct on you. It may not be explicitly on your mind, but at some level you're aware that that's part of the package you're choosing to sign up for.
Maybe rationalists should found more regular societies? In Germany, it is said that if three people come together, they found an association (it requires seven). The process is easy and common. It seems the US counterpart is an "501(c)(3) entity" and I have no clue how easy it is, but it shouldn't be difficult for the average member of this community. That would implicitly provide for some governance structure.
Creating a 501c3 is not the hard part. The hard part would be convincing a decentralized and informal community that they should accept the proposed central organization, including trusting it when it said who they should associate with.
(Which people are naturally wary of!)
Agree. I don't think it will work well when an informal community already exists. It probably works best when you found the community is small (seven members). After the fact, it might work for sub-communities with specific purposes, such as organizing events.
If the organization owns the place where people meet, the organization can put someone on a blacklist, and the local community can either meet at that place without the banned person, or coordinate on finding a different place to meet at -- which would be difficult if the best local coordinators are members of the organization.
After a conversation with ChatGPT and reading up on it, it seems South Korea is furthest in the process of a working due process system for cancelation and defamation dynamics.
https://en.wikipedia.org/wiki/South_Korean_cyber_defamation_law
Good or bad, I doubt it's a coincidence that this is in a society with more more stringent cultural contexts and assumptions about face and what it is or isn't ok to say and do in regards to other people's reputations.
I am precommitting to not engaging with any comments, because I am mostly offline and I think that is good
Statements looking like this are the parts I agree with the most! Politics and everything similar kind of suck.
Trying to formalize something subjective, and enforcing a absolute global conclusion to something inherently relative, is not a game worth playing. I'd even argue that not trying in the first place might be for the best. Just go full Taoism here.
handling non-criminal misconduct
Why would you do this? If you've been on the internet for over 15 years, you should have noticed that we went from moderating rules and laws, to moderating people based on impressions, feelings and moral values. This is how nautrality died, and how the internet became politicalized. It's also when the internet started growing stupid (probably because the average intelligence and autistic and nerdy tendencies fell over time)
I don't think that due process is impossible when it comes to crimes and concrete rules and laws. But when it comes to subjective things like being a "good person" or a "bad person", it's very difficult. To be frank, I think it's bad taste to try and force other people to adopt ones own opinions in the first place. Tribalism and herd dynamics is the very issue, not the solution, as those in the dynamic naively thinks (because they're converging towards a shared opinion, which they think is equal to solving the problem). The entire problem with echo-chambers are common discrimination is basically that holistic thinking is replaced by thinking in labels.
I think that if you have a community with reasonable people, none of this is a problem. And if you have a community with unreasonable people (which I define as the most common type of person, those prone to conformity and tribalism rather than individualism), then no system can help you. Unreasonable people don't even want fairness and justice in the first place, but revenge. They'll call their revenge "justice" of course, but this is only true from their experienced subjective reality / storytelling. It's a sort of delusion, but it's not inherently bad as those who are too disillusioned end up doing nothing, whereas those who truly care are able to influence the world.
Anyway, trying to solve problematic herd behaviour with a system or process is like trying to solve low intelligence with better education, it won't work. Social dynamics are self-balancing, if somebody is an unlikable person, they will become disliked over time naturally. Trying to impose artificial systems to overwrite natural systems is where the modern society fails, for you can't design complex processes/systems and just have them work. These things work bottom-up, not top-down.
Social dynamics are self-balancing, if somebody is an unlikable person, they will become disliked over time naturally.
I think that doesn't count as self-balancing unless that's the only way to become disliked.
What disturbs this balance is only people trying to control the world rather than letting things follow a natural flow. But (and this might be a lazy tautology), that too is self-balancing in a sense, as a community which cannot hold itself together will split in two or destroy itself.
If something requires constant effort, it's a sign that you're trying forcing something into an unnatural state. After reaching this unnatural state, you have a complex problem with many factors, which is why I argue "Avoid getting in the state in the first place, or just let the thing collapse".
In any case, I think we're trying to solve the wrong problem here. If somebody hasn't broken the law, I don't want to brainstorm how we can punish them and get public opinion on our side in hating them. On the other hand, if somebody is disliked by everyone, then you can argue their innocence all that you want, peoples feelings are unlikely to change by logical argument
I can't help but think a lot of this stuff is caused by an ontology mismatch. People encounter dynamics where they feel stuck or harmed, and they want there to be norms that prevent those dynamics, but norms operate on the level of behavior, which is typically too adaptable and nuanced to really be the source of the problems people encounter. You basically have to directly manage your community on the basis of traits that create patterns of behavior, rather than on the basis of the behavior itself. But this requires a comprehensive ontology of harm-relevant traits to make decisions on the basis of.
(This is related to the fact that Bayesian inference fails if you lack the grain of truth/realizability assumption. If your prior assumes some people Are Bad but that's not really how people differ, then your probability estimates for whether someone Is Bad are not gonna converge based on evidence.)
This sort of ontology is usually constructed in a messy way that tends to be pretty controversial among rationalists, and it tends to be communicated very poorly, which tends to make it hard to apply it in practice because every part is going to recursively seem wrong.
I think I know better (or at least, more rationalist-compatible) ways to construct and communicate such ontologies, but I have a hard time finding people who are interested in committing a lot of work to testing it with me, and also I've been persuaded that this isn't the only problem and that there are other problems too, including some of what you stated in the article. In addition to the ontology mismatch issue, two of the problems that I think are most significant are 1) lack of information, 2) corruption of justice processes.
Problems 1 and 2 seem worsened a lot by applying it to conflict rather than to other subjects, so I'm inclined to give up advertising my method for justice processes. Still, if anyone wants to give it a go and is willing to commit a ton of work into it from their community, please reach out to me.
1I don't understand what you're trying to say - would you mind trying to illustrate what you mean?
I wouldn't mind in principle, but it is extremely compressed, and kind of stitched together. As in, while I have examples of individual bits and pieces of my claims, I don't have any examples that go end-to-end. Instead I derived bits of the theory from the different examples and then stuck those theory-bits together into an overall framework.
So basically I can zoom in on different bits but I don't have time to zoom in on it all at once. (I am working writing it all up at once, but that's not ready yet. Partly I'm also delaying because I prefer searching for examples that go end-to-end and because I'm still learning new nuances and techniques.)
TLDR: This is an long metaphor to draw parallels between the Hanseatic League and the broader EA/LW communities. It is OK to not be a {corporation, societas, collegium, universitus} with common/top-down/bottom-up violence-monopolizing system. The price to implement a resolution system people would find satisfactory might be too high.
The fact it is hard to resolve conflict is because it is an integral part of the bargain, not an isolated bug. I personally don't want us to become a chimera with nine heads - a chimera for the sole purpose so we can utter that "we are a state".
“the [Efficax Altruismus] is not a societas: (a company) for it knows neither a common ownership of goods nor shared ownership of the good, since in the [Efficax Altruismus] there is no joint ownership; nor is it a company formed for certain commercial transactions, since in the [Efficax Altruismus] each individual makes transactions for himself, and the profit and loss falls on each individual…
It is also not a collegium (a college)….since it is formed from separate [communities]. It is also not a universitas (a corporate body), because…for it is required that it has property, a common treasure, a common seal, a common syndicus and a recognised leader.
“the [Efficax Altruismus] is … a firm alliance of many cities, towns and communities for the purpose of ensuring that enterprises on water and on land have the desired and favourable success and that effective protection is provided against pirates and highwaymen, so that the merchants are not deprived of their goods and valuables by their raids.” - The Hanse in Medieval and Early Modern Europe, I think...
Perfect due process is impossible for the reasons you describe. But there's a difference between "not perfect" and "egregiously bad", and if you focus too narrowly on the inability to make the process perfect, people are going to get away with processes that are egregiously bad.
If you wrote this in February, it preceded the Nonlinear accusations. From what I can tell from what I read here, they're a lot closer to "egregiously bad" than to "not perfect". Do they change your opinion of due process to any extent?
I am not a community organizer, but if I was, I would just edict the MAD rule : if any dispute can’t be reasonably settled (in a way that would escalate to a panel), both the plaintiff and the defendant are kicked out of the community. The ratio of assholes to victims being low (hopefully), frame it as a noble sacrifice from the victim to keep the rest of the community safe (whoever is the actual victim here). "Some of you may die, but it’s a sacrifice I’m willing to make".
True, you’re slightly disincentivizing the reporting of true abuses (but not so much, since you’re saving a lot on the trauma of investigation). But you’re also heavily disincentivizing both false accusations and "I’m a good manipulator so I’m confident I will be able to get away with it". Add to it the man-hours saved, and I’m pretty confident it is a net gain — as long as you can enforce it.
As a bonus, if a dispute end up with community bans, you can be reasonably sure that the plaintiff was the victim. But of course, that only works if you don’t go from that observation to concluding "therefore the victim is un-banned". A noble sacrifice, really, in service of both community and truth. A sacrifice nonetheless.
1This is how adults deal with fights among children. The adults don't care who started it, who did what to whom, justice, fairness, or anything at all but getting them to stop bothering the adults.
It is not a way to deal with adults.
There are a bunch of problems with this even if implemented in good faith, but an obvious attack is that you convince a friend external to the community to (meet the minimum qualifications for membership and) accuse your target. This gets your target and your friend banned without investigation, but of course your friend does not care about getting banned.
both the plaintiff and the defendant are kicked out of the community
This is unironically a great idea. It doesn't hinge on annoying questions of evidentiary standards, mens rea, seriousness of the charge, conflicts of interest etc. The idea that you can take anyone down, no matter how influential or powerful, if you're willing to go down with them really appeals to me.
as long as you can enforce it.
Yeah, you probably can't.
1The idea that you can take anyone down, no matter how influential or powerful, if you're willing to go down with them really appeals to me.
I think this is a mistake; it's easy to twist people out of shape such that they take themselves down and one of your enemies.
This is prevalent in elite human systems e.g. among corporate executives, and it's instrumentally convergent so smart psychos will discover opportunities for it here too, and it's also highly exploitable by tech companies and intelligence agencies if anyone working there ever decides they'd like AI safety out of the way.
You're right that this would open up new avenues for inter-elite conflict. That's the tradeoff for sharply curbing elite abuse of the less powerful.
I think this was good commentary, and I would never have noticed the exploit without it. For the record, I upvoted.
Disagreement votes don't lower your karma, do they? I (strongly) disagreed, but I don't want to even marginally contribute to you getting rate limited. Either way, I have now more than evened it out.
1I've never been rate-limited, not even on EAforum where I get downvoted a lot, so I worry more about the social status/brand reputation loss from people seeing my comments with any kind of negative karma attached to them.
The idea that you can take anyone down, no matter how influential or powerful, if you're willing to go down with them really appeals to me.
Suicide bombing?
The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.
Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?
As a counter example I think the EA community health team does better than I would expect. A huge global community and they are relatively trusted and the community takes responsibility for itself. Like I don't know that I'd say it does well but I think I'd be hard pressed to find an analogous community that has a better such function.
Feels like they have a better batting average than rationalists in dealing with issues like this (I guess at least partly cos events are more central to EA than rationalism). Many other communities seem to have no responsibility at all - many political groups seem rife with issues but noone taking responsibility over them.
I agree overall that this is a really hard problem, though I suppose I am more optimistic than OP.
I have had two friends thrown out of the community without due process, and it was absolutely awful for them. For one of them, I was never told enough about the situation to make my own judgment, but the other one was thrown out basically just because a couple powerful people didn't like his vibes.
I've also learned: It is easier to remove good people from your community than to remove people who are aggressive, manipulative, narcissistic, or psychopathic, because the former will remain reasonable throughout the process, while the latter will fight dirty.
This is why it's so important to never, ever, EVER give people quotas for purging, or rewarding people in any way for racking large numbers of purge cases, no matter how good they make each purge case look.
Goodharting will typically take place if you stack the incentives like that.
1The thing you quoted doesn't imply that there were any quotas or rewards, or metrics being Goodharted. (Definitely agree that quotas or rewards for "purging" is a terrible idea.)
1I have the impression that it is a general problem with public prosecutors (source: cultural osmosis). So if one were to imagine creating someone whose job is analogous to prosecuting criminals, one would have to be very careful about the incentives one sets up for that person, and it's not obvious that rationalists could walk in and immediately solve it better than society has managed to do with prosecutors.
2I assume no one will read this comment
assuming the problem is really intractable and the current panel process is the best available solution, then the standard solution is to put up a [scapegoat]
i.e. civil servant do not want to/ not able to do something for someone, instead of saying "this is my judgement", point to an other entity [e.g. code of conduct, boss, etc], and deflect the blame. the point is not to deflect the blame though, but to keep on functioning despite having to make unpopular decisions.
I assume that chatGPT would make an excellent [scapegoat]
feed all the gathered evidences to ChatGPT, ask it for judgement [with the appropriate precondition: " you are a wise and benevolence judge, etc"], if it agree with the panel decision, then when the inevitable blow back happen you can point to chatGPT and said it agreed with you and it is obviously unbiased
if it disagreed with the panel decision then it would be a sanity check, the panel should find more evidence or double check their reasoning, since ChatGPT can serve as stand in for the average Joe who read all the evidence, if it is not convinced you do not have a convincing case.
Not a lawyer, but the "can't explain your reasoning" problem is overblown. Just need to be very diligent in separating facts from the opinions and findings of the panel. There is a reason every report of that sort done professionally sounds the particular flavor of stilted that it does.
"Our panel found that <accused> did <thing>" <- potential lawsuit, hope you can prove that in court. You're not a fact finder in a court of law, speak as if you are at your own peril.
"Our panel was convened to investigate <accusation> against <accused>. Based on <process>, we believe the accusation credible and recommend the following: ..." <- A OK
"Our panel was convened to investigate <accusation> against <accused>. Based on <process>, we were unable to corroborate the accusation and cannot recommend action at this time." <- A OK
"person X said Y, I did/didn't believe them" is basically always fine, provided X actually said Y. Quoting someone else's statement is well protected, and you're entitled to your opinions. The trouble happens when your opinions/findings/beliefs are stated as facts about what happened instead of facts about what you believe and what evidence you found persuasive.
There's also nothing stopping the panel from saying "we heard closed door testimony...", describe the rough topic, speakers relation to the inquiry, and the degree to which it was persuasive.
There's a more general concern here for running organizations where anyone can sue anyone at any time for any reason, merit or no. If one allows the barest hint of a lawsuit to dictate their actions, that too becomes another vector through which they can be manipulated. Perhaps a better thing to aim for is "don't do anything egregious enough a lawyer will take it on contingency", use additional caution if the potential adversary is much better resourced than you (and can afford sustained frivolous litigation).
I wrote this entire post in February of 2023, during the fallout from the TIME article. I didn't post it at the time for multiple reasons:
But after sitting on it for a full year, I still think it's worth posting, so here it is. The only edits I have made since February 16th, 2023, were to add a couple interstitial sentences for clarity, and change 'recent articles' to 'articles from February 2023'. So, it's not intended to be commenting on anything more recent than that.
I am precommitting to not engaging with any comments, because I am mostly offline and I think that is good. I probably won't even look at this post again for several weeks. Do what you will. Here is the post:
Note: I am erring on the side of not naming any names in this article. There is one exception for the sake of clarity.
In my time overseeing the global rationalist community, living in the Bay community, and just generally being a person, I've seen a lot of people face up to complicated conflicts.
People often get really mad at each other for mishandling these cases, and will sometimes publicly point to these failures as reasons to condemn a person or group. However, I challenge you to point to a single entity in the world that has figured out a process for handling non-criminal misconduct that you would be happy with no matter whether you were the aggrieved or the accused party. Maybe such a thing exists, but if so I have not heard of it.
This post is a survey of the different ways that people try to resolve community conflicts, and the ways that each of them fail.
Committees/panels
In cases of major conflict or disagreement, it often seems like the right thing to do to convene a panel of impartial judges and have them hear all the evidence. I personally know of at least seven specific cases of this happening in the rationalist community. Here are some of the problems with this approach.
Investigations eat up hundreds of person-hours
The case I'm most familiar with has been investigated four different times, by different people and from different angles. Five separate reports have been written. At time of writing the situation has dragged out for three full years, and it's consumed over 100 hours of my time alone, and who knows how much time for the other like 30 people involved.
You might think "holy shit, at that point who even cares, this is obviously not worth all those precious life hours that those 30 people will never get back, just ban the guy." I'm inclined to agree, but unfortunately:
Panels generally don't have much real ability to enforce things
If the members of your community don't agree with your decision to ban someone, you can't force them to abide by your decision.
Here are the actions available to you:
Here are things you cannot do:
Panels act like they are courts of law
In a court of law, you are presumed innocent unless and until you can definitively be proven guilty of a specific crime. But this is far too strict a standard to use in community dispute — well-kept gardens die by pacifism, and you shouldn't have to keep someone in your community if they're causing problems, even if you can't find definitive proof that they did some particular thing that everyone would agree is really bad.
On the opposite extreme, in #MeToo you are presumed guilty as soon as there's an allegation against you. The intent is to counteract the perceived phenomenon of assault allegations being dismissed by default, but obviously believing 100% of allegations uncritically would go too far in the other direction. In 2014, Scott estimated that "greater than 0.3% of men get falsely accused of rape sometime in their lives, and the most likely number is probably around 3%."
I think the best you can do is to make a holistic judgment, not relying on any one piece of evidence. You should be able to pass judgment on someone if they're just generally agreed to be terrible by many people, even if they haven't done any one provable thing that is really egregious.
But panelists often feel like they have to adhere to a really high standard of evidence, because:
Panels often lack a secure sense of legitimacy
If the community disagrees with your ruling, they may not only refuse to adhere to it, but also decide that they no longer trust you. In the worst case, they may decide you need to be removed from power or even socially destroyed. I've seen this rip communities apart. Angry constituents threaten panelists' reputations and livelihoods, and the reputations of their friends and allies. People may divide into factions that never forgive each other.
With this hanging over you, it's hard to not take it into account when making your ruling.
Favorable rulings can lend legitimacy to bad actors
If the panel ends up officially condoning the behavior of a bad actor — something they are likely to do even if they personally think the person is terrible, if they believe they need standards for evidence and conviction comparable to those in an actual court of law — that gives the bad actor legitimacy, and can allow them to do more damage.
Extended investigations are extremely stressful for all parties involved
More than 50% of the reason I stepped down as global organizer was that I became entangled in all these horrible conflicts with very angry people on all sides.
The worst conflict dragged out for about 8 months. I was involved first as an investigator, then, after I passed my judgment and the accused parties took issue with my decision and my process, I became the subject of an investigation where the original accused party accused me of unfair conduct, abuse of power, and unfitness for leadership. They took their accusations both to the person they believed to be my boss, and to the people I oversaw, trying to destroy those people's trust in me. I received dozens or sometimes hundreds of messages a day from them, and was accused of stonewalling and dishonestly refusing to respond when I had really legitimate excuses like “my sister is in the hospital with internal bleeding", or "I'm on a transatlantic flight and don't have internet" or “it is literally the middle of the night where I am" or “this has been going on for four weeks and I want to live my life”.
At one of the four heights of the conflict, all I did was cry, complain to anyone who would listen, drink alcohol, and then cry more. None of this was worth it. It burned me out horribly, burned out at least two other organizers, and made me and dozens of other people miserable.
---
After the mishandling of the Brent Dill case (details below), one of the three panelists essentially went into hiding, avoiding all social events for months.
---
The aggrieved parties are often dredging up trauma and sometimes become too exhausted to continue.
There is often no way to find the objective truth
Much of the time, it just comes down to one person's word against another, and you often have no way of knowing who to trust.
Other approaches
Throwing out due process is also problematic
I have had two friends thrown out of the community without due process, and it was absolutely awful for them. For one of them, I was never told enough about the situation to make my own judgment, but the other one was thrown out basically just because a couple powerful people didn't like his vibes.
I've also learned: It is easier to remove good people from your community than to remove people who are aggressive, manipulative, narcissistic, or psychopathic, because the former will remain reasonable throughout the process, while the latter will fight dirty.
Codes of conduct can't save you
Sometimes people suggest that maybe you could avoid all this litigation if you just make a really good and clear code of conduct and then throw people out if they violate it. That would be nice, but it doesn't work in practice for rationalist communities, mostly because:
You can't impose strictures unilaterally / without the consent of the governed
When joining a workplace, a university, or a club sport, you are opting into a code of conduct, including the part where there's an entity that enforces that code of conduct on you. It may not be explicitly on your mind, but at some level you're aware that that's part of the package you're choosing to sign up for.
Workplaces can have codes of conduct, because they are (often) neatly circumscribed social environments, and have a clear purpose. The same is often true of recreational sports. It is not true of the rationality community.
At a local meetup group, where the organizer is widely liked and recognized as the group leader, the leader may have the power to exclude people from their events – especially if the the meetings are held at a private location like the organizer's house. But the rationalist community also encompasses various workplaces and group houses, which have various amounts of power to exclude people.
The lack of a single institution makes the problem more difficult, but it's also not something that can be easily changed (nor do I think it should be changed).
Introducing codes of conduct creates weird dynamics
The most common response I get from organizers when I bring up codes of conduct is: "It's true that we don't want women to be driven off by a bunch of awkward men asking them out, but if we make everyone read a document that says 'Don't ask a woman out the first time you meet her', then we'll immediately give the impression that we have a problem with men awkwardly asking women out too much — which will put women off anyway."
This is a good point and I don't know how to get around it, so I generally just don't recommend formal codes of conduct.
It's difficult to write rules that will actually lead to the thing you want
As a (maybe weak) example, most people are assuming some implicit code of conduct in everyday interactions, like "Don't be a sexual predator", "Don't be a dickhead", and "Don't ruin other people's lives." But when you try to litigate these things, you run into problems like:
Don't be a sexual predator -> What constitutes predation? What if I thought it was consensual and they didn't? What if it was technically statutory rape but the younger party was 17 and enthusiastically consented and has never regretted it? What if I didn't do anything that looks wrong if you go strictly by the book, but I nonetheless traumatized a bunch of people and feel no remorse about it?
Don't be a dickhead -> What if 90% of people think I'm dickhead but I can find four witnesses who really like me? What if I provide a lot of value to your community; at what point am I so valuable that I can get away with being a dickhead? What if nonstandard social interactions are part of the way I provide value?
Don't ruin other people's lives -> I didn't ruin their lives, they're just oversensitive. Also, throwing me out of the community would ruin my life, therefore, you can't do it.
Going to the authorities
In multiple articles from February 2023, it's suggested that a responsible course of action in sexual misconduct cases would involve encouraging women to report their experiences to the police. They also mention Title IX as an example of what we should be aspiring to. I think both of these are things that just sound nice as soundbites, but are actually not good suggestions.
I'm including this section mostly in response to those articles, but it will probably be obvious that I don't actually take these suggestions seriously.
Title IX
As a student at a US university who has experienced sexual harassment or assault, you have recourse to the university's Title IX Coordinator. If you choose to report, the university's Title IX Hearing Panel will investigate the case by talking to you, the accused, and any other relevant parties such as witnesses.
If this sounds like my description of panels above, that's because it is pretty much the same thing. It has all the pitfalls of panels, with the additional downsides that the people are more likely to be strangers to you and are less likely to be aligned with your interests. (For one thing, the college has a vested interest in making sure you do not report sexual assault to the police.)
I had several friends in college go to the school authorities after being sexually assaulted. I do not think any of them were happy with either the experience or the outcomes.
I remember reading an article (though I can't find it; it was nine years ago) about one of my friend's cases, where the assailant's father came in and said that the whole thing was preposterous and my friend was just a jilted lover who wanted attention. I don't think an ideal process would open complainants up to public defamation by the fathers of their assailants.
Other prominent incidents that happened at my school:
The police
It's confusing to me that this is even a suggestion, because I frankly don't see what the police are expected to do in most cases. If a person has been sexually assaulted in a way that left them with both injuries and the perpetrator's DNA on them, and they want to press charges that might result in a criminal sentencing, then that seems like the kind of thing the police are equipped to handle. Otherwise, I don't see how going to the police does much good. I couldn't find reliable statistics (I found many but they were highly varied), but the general vibe seems to be that sexual assault cases are rarely prosecuted.
Maybe the idea is that if the police receive multiple accusations against a person, they will eventually take them seriously, and maybe that person will become a registered sex offender or something? I don't really know how this works but in any case it doesn't seem like an obviously good way to resolve disputes within a community.
This suggestion is also confusing because there's a lot of other messaging that the police are not to be trusted, so like, why should we go to them in this specific scenario?
HR departments
HR departments exist to serve the company, not the employees, and so they are, in general, morally corrupt.
Case study
Let's take a look at the case of Brent Dill. This is the only name I will be naming in this post, because I thought that obfuscating it would do little good and just be unnecessarily confusing.
I'm including this case study because I think a concrete example is a lot more helpful than just saying "Look, we've tried justice panels and they didn't work as intended, take my word for it." It's also a case where most of the important details are already public, and where the case is widely considered to be closed.
Relevant reading:
Whisper network
When I arrived in the Bay in June of 2017, fresh out of college, Brent was still in good standing in the community. I knew many people who were friends with him or considered themselves part of his circle.
However, even then, there were plenty of people who found Brent to be toxic and dangerous — it was not a secret or even a controversial statement that Brent had a lot of darkness in him; I'd say that pro-Brent and anti-Brent people mostly just differed in how they assessed the expected value of interactions with him.
As I was a young woman and a new member of the community, multiple people were quick to tell me to stay away from Brent; after meeting him I immediately agreed with that assessment and went on to join in warning other people about him.
CFAR panel
In 2017, CFAR convened a panel of three people (only one of whom was a CFAR employee) to hear community disputes. This panel was called the Alumni Community Dispute Council, or ACDC for short.
In January of 2018, one of Brent's ex partners came to ACDC with allegations of a pattern of abuse by Brent. ACDC investigated this case for many hours, and in April, they privately recommended to CFAR that no action be taken and that Brent remain a community member in good standing.
I knew one of the three ACDC panelists well at the time, and that person did not agree with the conclusion that Brent should remain in the community. They were the first to warn me and many other young women to stay away from Brent, and I had previously seen them try pretty hard to block Brent from attending a private CFAR event. The fact that this person ended up party to a pro-Brent ruling indicates that something in the process went very wrong.
Since it is now widely agreed that kicking Brent out of the community was the correct course of action, I think it's reasonable to ask how this ruling could have happened.
I think the ACDC panelists were well-intentioned, but as CFAR's Executive Director put it in CFAR's Mistakes Regarding Brent, they were "in over their heads." Panels may be equipped to hear cases of clear-cut assault, but it's much more difficult to make a ruling on a potentially abusive but nominally consensual relationship between two adults.
In addition, this case had some factors that made it difficult beyond just the inherent complexity of figuring out interpersonal conflicts between adults. Specifically:
These factors are not unique to Brent. In most of the truly complex and protracted cases that I'm familiar with, at least one party has been significantly manipulative. Many well-intentioned people, even those who are good at dealing with conflict or have other qualities that might make them a good fit for a conflict resolution panel, are still susceptible to this kind of manipulation. It's also just very difficult to come to a conclusion when not all parties are engaging in good faith.
Medium posts
In September of 2018, two of Brent's former partners and one of his former friends posted publicly about their experiences with him on Medium, under the name Mittens Cautious. The case was thereby brought before the court of public opinion, and public opinion came down firmly on the side of Mittens Cautious.
When people found out about ACDC's previous ruling on Brent, many were appalled that ACDC had seen the evidence laid out in the Medium posts and ruled that it was okay for Brent to continue on like that. ACDC was disbanded within a week of the first Medium post, and people publicly questioned whether they could trust CFAR or the members of ACDC going forward.
REACH panel
Immediately after the Medium posts went up, the Berkeley REACH (a rationalist & EA community center that existed at the time) announced the formation of a panel intended to apply actual due process to the Brent case. This panel was elected by a vote of REACH stakeholders, and carried out an investigation over the next three months.
In December, the panel released a statement saying that they recommended that Brent be banned from REACH, but that, following legal advice they'd received, they would not be publicly sharing their reasoning. It's not mentioned in the linked statement, but I believe the concrete worry was that publishing a statement that casts someone in a negative light opens you up to a libel lawsuit. So, the community as a whole gained very little from this repeated investigation.
When does community resolution work?
I have seen some local rationalist communities handle conflicts and bans relatively gracefully. These communities:
In those communities, processes for hearing interpersonal complaints can sometimes actually work. Some groups (generally smaller ones) elect a benevolent dictator; others let all their members vote to elect a panel.
In a sub-Dunbar community, everyone (approximately) knows each other, which makes it easier to trust the decisionmakers and the process that chose them.
Also, in a meetup-based community, the decisionmakers have some real enforcement authority, if they find that sanctions are necessary. They can't stop anyone from being friends with the accused, but they definitely can exclude them from meetups and online spaces — and if the meetups (and online spaces) are the heart of the community, this is a pretty effective way of excluding the person from the community as a whole.
However, having a small, structured community is not sufficient to ensure that this will work.
What would be the elements of a good system?
Now that we've seen all the ways that things can go wrong, it should be easy to reverse those and see how things could go right! If we can't look at all the factors and then figure out the optimal course of action from first principles, how can we call ourselves rationalists?!
That's a joke; there's a reason this post is called "The impossible problem of due process". That said, I think that a good system for community resolution of conflicts would probably have the following elements: