This post is written in a personal capacity. All opinions expressed are solely those of the author. I. Executive Summary There’s a mismatch that I’d like to correct between: 1. the reasons why superintelligence is dangerous, and 2. the categories we use to describe how dangerous superintelligence is. The most...
I gave a talk this year at LessOnline about how to make AI governance research more relevant to policymakers (videos available here), and the subtopic from that talk that got the most audience engagement was how to turn a vague moral sentiment into a concrete policy proposal. During the talk,...
Previously in this sequence, I estimated that we have 3 researchers for every advocate working on US AI governance, and I argued that this ratio is backwards – we need the political power provided by advocates to have a chance of preventing misaligned superintelligence. A few researchers might be useful...
INTRODUCTION The Story So Far In my first three posts in this sequence, I estimated that we have 3 researchers for every advocate working on US AI governance, and I argued that this ratio is backwards. Even the best research offers only modest and indirect support for advocacy, and research...
In previous posts in this sequence, I laid out a case for why most AI governance research is too academic and too abstract to have much influence over the future. Politics is noisy and contested, so we can’t expect that good AI governance ideas will spread on their own –...
In my previous post in this series, I estimated that we have 3 researchers for every advocate working on US AI governance, and I argued that this ratio is backwards. When allocating staff, you almost always want to have more people working on the more central activity. I argued that...
In my previous post in this series, I explained why we urgently need to change AI developers’ incentives: if we allow the status quo to continue, then an AI developer will recklessly deploy misaligned superintelligence, which is likely to permanently disempower humanity and cause billions of deaths. AI governance research...