Paranoia: A Beginner's Guide
People sometimes make mistakes. (Citation Needed) The obvious explanation for most of those mistakes is that people do not have access to sufficient information to avoid the mistake, or are not smart enough to think through the consequences of their actions. This predicts that as decision-makers get access to more information, or are replaced with smarter people, their decisions will get better. And this is substantially true! Markets seem more efficient today than they were before the onset of the internet, and in general decision-making across the board has improved on many dimensions. But in many domains, I posit, decision-making has gotten worse, despite access to more information, and despite much larger labor markets, better education, the removal of lead from gasoline, and many other things that should generally cause decision-makers to be more competent and intelligent. There is a lot of variance in decision-making quality that is not well-accounted for by how much information actors have about the problem domain, and how smart they are. I currently believe that the factor that explains most of this remaining variance is "paranoia". In particular the kind of paranoia that becomes more adaptive as your environment gets filled with more competent adversaries. While I am undoubtedly not going to succeed at fully conveying why I believe this, I hope to at least give an introduction into some of the concepts I use to think about it. A market for lemons The simplest economic model of paranoia is the classical "lemon's market": In the classical lemon market story you (and a bunch of other people) are trying to sell some nice used cars, and some other people are trying to buy some nice used cars, and everyone is happy making positive-sum trades. Then a bunch of defective used cars ("lemons") enter the market, which are hard to distinguish from the high-quality used cars since the kinds of issues that used cars have are hard to spot. Buyers adjust their

Roko has IMO kind of obviously gone off the rails and this feels to me like a success case of the system. Like, I think it's more likely than not that we would ban Roko if he kept commenting, just based on past experiences with him.
I agree with some of the general considerations otherwise, but this specific case feels like a success.