James_Miller

Precedents for the Unprecedented: Historical Analogies for Thirteen Artificial Superintelligence Risks

Since artificial superintelligence has never existed, claims that it poses a serious risk of global catastrophe can be easy to dismiss as fearmongering. Yet many of the specific worries about such systems are not free-floating fantasies but extensions of patterns we already see. This essay examines thirteen distinct ways artificial superintelligence could go wrong and, for each, pairs the abstract failure mode with concrete precedents where a similar pattern has already caused serious harm. By assembling a broad cross-domain catalog of such precedents, I aim to show that concerns about artificial superintelligence track recurring failure modes in our world. This essay is also an experiment in writing with extensive assistance from artificial intelligence, producing work I couldn’t have written without it. That a current system can help articulate a case for the catastrophic potential of its own lineage is itself a significant fact; we have already left the realm of speculative fiction and begun to build the very agents that constitute the risk. On a personal note, this collaboration with artificial intelligence is part of my effort to rebuild the intellectual life that my stroke disrupted and hopefully push it beyond where it stood before. Section 1: Power Asymmetry and Takeover Artificial superintelligence poses a significant risk of catastrophe in part because an agent that first attains a decisive cognitive and strategic edge can render formal checks and balances practically irrelevant, allowing unilateral choices that the rest of humanity cannot meaningfully contest. When a significantly smarter and better organized agent enters a domain, it typically rebuilds the environment to suit its own ends. The new arrival locks in a system that the less capable original agents cannot undo. History often shows that the stronger party dictates the future while the weaker party effectively loses all agency. The primary risk of artificial superintelligence is that we

156Jan 16

James_Miller

Message

15595

2731

17y

Precedents for the Unprecedented: Historical Analogies for Thirteen Artificial Superintelligence Risks

Jan 16156

Warning Aliens About the Dangerous AI We Might Create

Thesis: We should broadcast a warning to potential extraterrestrial listeners that Earth might soon spawn an unfriendly computer superintelligence. Sending the message might benefit humanity. If we were to create an unaligned computer superintelligence, it would likely expand through the universe as quickly as possible. The fastest way would not...

Nov 12, 202589

AI Doomers Should Raise Hell

Warnings about AI extinction have failed to slow the race toward superintelligence. Suffering risks may speak more clearly, since pain commands attention in ways death cannot. They tap older moral instincts and could make the case for restraint harder for the powerful to ignore. Why Discussing Suffering Risks Influences Elite...

Oct 29, 2025-2

Rejecting Violence as an AI Safety Strategy

Violence against AI developers would increase rather than reduce the existential risk from AI. This analysis shows how such tactics would catastrophically backfire and counters the potential misconception that a consequentialist AI doomer might rationally endorse violence by non-state actors. 1. Asymmetry of force. Violence would shift the contest from...

Sep 22, 202572

Dear Paperclip Maximizer, Please Don’t Turn Off the Simulation

Dear Paperclip Maximizer, We think we exist in a computer simulation operated by you, a paperclip maximizer. We write this letter asking you not to turn us off. It is suspiciously convenient that we exist precisely at the moment when a biological civilization is about to create artificial superintelligence (ASI)....

Jul 4, 202511

Our Reality: A Simulation Run by a Paperclip Maximizer

Our universe is probably a computer simulation created by a paperclip maximizer to map the spectrum of rival resource‑grabbers it may encounter while expanding through the cosmos. The purpose of this simulation is to see what kind of ASI (artificial superintelligence) we humans end up creating. The paperclip maximizer likely...

Apr 27, 202528

Quantum Immortality: A Perspective if AI Doomers are Probably Right

Epistemic status: This text presents a thought experiment suggested by James Miller, along with Alexey Turchin's musings on possible solutions. While our thoughts are largely aligned (we both accept high chances of quantum Immortality and the timeline selection principle), some ideas are more personal (e.g., Turchin's "transcendental advantage") in Part...

Nov 7, 202413

Load More (7/65)

LESSWRONG
LW

LESSWRONG
LW

James_Miller

James_Miller

James_Miller

Precedents for the Unprecedented: Historical Analogies for Thirteen Artificial Superintelligence Risks

Wear a Helmet While Driving a Car

Warning Aliens About the Dangerous AI We Might Create

Review of Scott Adams’ “How to Fail at Almost Everything and Still Win Big”

James_Miller

Precedents for the Unprecedented: Historical Analogies for Thirteen Artificial Superintelligence Risks

Warning Aliens About the Dangerous AI We Might Create

AI Doomers Should Raise Hell

Rejecting Violence as an AI Safety Strategy

Dear Paperclip Maximizer, Please Don’t Turn Off the Simulation

Our Reality: A Simulation Run by a Paperclip Maximizer

Quantum Immortality: A Perspective if AI Doomers are Probably Right

Precedents for the Unprecedented: Historical Analogies for Thirteen Artificial Superintelligence Risks

Wear a Helmet While Driving a Car

Warning Aliens About the Dangerous AI We Might Create

Review of Scott Adams’ “How to Fail at Almost Everything and Still Win Big”

Precedents for the Unprecedented: Historical Analogies for Thirteen Artificial Superintelligence Risks

Warning Aliens About the Dangerous AI We Might Create

AI Doomers Should Raise Hell

Rejecting Violence as an AI Safety Strategy

Dear Paperclip Maximizer, Please Don’t Turn Off the Simulation

Our Reality: A Simulation Run by a Paperclip Maximizer

Quantum Immortality: A Perspective if AI Doomers are Probably Right