habryka

Paranoia: A Beginner's Guide

People sometimes make mistakes. (Citation Needed) The obvious explanation for most of those mistakes is that people do not have access to sufficient information to avoid the mistake, or are not smart enough to think through the consequences of their actions. This predicts that as decision-makers get access to more information, or are replaced with smarter people, their decisions will get better. And this is substantially true! Markets seem more efficient today than they were before the onset of the internet, and in general decision-making across the board has improved on many dimensions. But in many domains, I posit, decision-making has gotten worse, despite access to more information, and despite much larger labor markets, better education, the removal of lead from gasoline, and many other things that should generally cause decision-makers to be more competent and intelligent. There is a lot of variance in decision-making quality that is not well-accounted for by how much information actors have about the problem domain, and how smart they are. I currently believe that the factor that explains most of this remaining variance is "paranoia". In particular the kind of paranoia that becomes more adaptive as your environment gets filled with more competent adversaries. While I am undoubtedly not going to succeed at fully conveying why I believe this, I hope to at least give an introduction into some of the concepts I use to think about it. A market for lemons The simplest economic model of paranoia is the classical "lemon's market": In the classical lemon market story you (and a bunch of other people) are trying to sell some nice used cars, and some other people are trying to buy some nice used cars, and everyone is happy making positive-sum trades. Then a bunch of defective used cars ("lemons") enter the market, which are hard to distinguish from the high-quality used cars since the kinds of issues that used cars have are hard to spot. Buyers adjust their

350Nov 13, 2025

habryka

Message

Running Lightcone Infrastructure, which runs LessWrong and Lighthaven.space. You can reach me at habryka@lesswrong.com.

(I have signed no contracts or agreements whose existence I cannot mention, which I am mentioning here as a canary)

56100

1923

287

6193

118

Dario Amodei – The Adolescence of Technology

Dario Amodei, CEO of Anthropic, has written a new essay on his thoughts on AI risk of various shapes. It seems worth reading, even if just for understanding what Anthropic is likely to do in the future. Confronting and Overcoming the Risks of Powerful AI There is a scene in...

Jan 26147

Lightcone is hiring a generalist, a designer, and a campus operations co-lead

Lightcone is hiring! We build beautiful things for truth-seeking and world-saving. We are hiring for three different positions: a senior designer, a campus manager, and a core team generalist. This is the first time in almost two years where we are actively hiring and trying to grow our team! Senior...

Jan 17118

Toss a bitcoin to your Lightcone – LW + Lighthaven's 2026 fundraiser

TL;DR: Lightcone Infrastructure, the organization behind LessWrong, Lighthaven, the AI 2027 website, the AI Alignment Forum, and many other things, needs about $2M to make it through the next year. Donate directly, send me an email, DM, signal message (+1 510 944 3235), or leave a public comment on this...

Dec 13, 2025313

Be Naughty

Context: Post #10 in my sequence of private Lightcone Infrastructure memos edited for public consumption. This one, more so than any other one in this sequence, is something I do not think is good advice for everyone, and I do not expect to generalize that well to broader populations. If...

Nov 22, 202599

Automate, automate it all

Context: Post #9 in my sequence of private Lightcone Infrastructure memos edited for public consumption. First, a disclaimer. Before you automate something, first see whether you can just not do the thing at all. Questioning the Requirements is a step that should always happen before you gleefully systematize a task....

Nov 19, 202574

Anthropic is (probably) not meeting its RSP security commitments

TLDR: An AI company's model weight security is at most as good as its compute providers' security. Anthropic has committed (with a bit of ambiguity, but IMO not that much ambiguity) to be robust to attacks from corporate espionage teams at companies where it hosts its weights. Anthropic seems unlikely...

Nov 18, 2025130

Aim for single piece flow

Context: Post #8 in my sequence of private Lightcone Infrastructure memos edited for public consumption. When you finish something, you learn something about how you did that thing. When you finish many things at the same time, you do not get to apply the lessons you learned from each of...

Nov 18, 2025120

Load More (7/219)

LESSWRONG
LW

LESSWRONG
LW

habryka

habryka

habryka

Paranoia: A Beginner's Guide

LessWrong's (first) album: I Have Been A Good Bing

My tentative best guess on how EAs and Rationalists sometimes turn crazy

Integrity and accountability are core parts of rationality

habryka

Dario Amodei – The Adolescence of Technology

Lightcone is hiring a generalist, a designer, and a campus operations co-lead

Toss a bitcoin to your Lightcone – LW + Lighthaven's 2026 fundraiser

Be Naughty

Automate, automate it all

Anthropic is (probably) not meeting its RSP security commitments

Aim for single piece flow

Paranoia: A Beginner's Guide

LessWrong's (first) album: I Have Been A Good Bing

My tentative best guess on how EAs and Rationalists sometimes turn crazy

Integrity and accountability are core parts of rationality

Dario Amodei – The Adolescence of Technology

Lightcone is hiring a generalist, a designer, and a campus operations co-lead

Toss a bitcoin to your Lightcone – LW + Lighthaven's 2026 fundraiser

Be Naughty

Automate, automate it all

Anthropic is (probably) not meeting its RSP security commitments

Aim for single piece flow