An AI risk argument that resonates with NYTimes readers
Ezra Klein of NYT put out a surprisingly sympathetic post on AI risk in the Sunday edition. It even quotes Paul Christiano and links back to LessWrong! But what I'm actually here to talk about is the top reader-recommended comment on the article as of Sunday 11pm UTC: > Dwarf Planet > > I wonder how many of these AI researchers have children. What Ezra describes here is what I see every day with my teenager. Of course, no one understands teenagers, but that's not what I mean. I taught my daughter to play chess when she was very young. I consider myself a reasonably good player, and for many years (as I was teaching her), I had to hold myself back to let her win enough to gain confidence. But now that she is thirteen, I suddenly discovered that within a span of weeks, I no longer needed to handicap myself. The playing field was level. And then, gradually and then very suddenly, she leapt past my abilities. As with AI, could understand the broad outlines of what she was doing--moving this knight or that rook to gain an advantage--but I had no clue how to defend against these attacks. And worse (for my game, at least), I would fall into traps where I thought I was pursuing a winning hand but was lead into ambush after ambush. It was very humbling: I had had the upper hand for so long that it became second nature, and then suddenly, I went to losing every game. > > As parents, we all want our children to surpass us. But with AI, these "summoners" are creating entities whose motives are not human. We seem to be at the cusp of where I was before my daughter overtook me: confident and complacent, not knowing what lay ahead. But, what we don't realize is that very soon we'll begin to lose every game against these AIs. Then, our turn in the sun will be over. Generally NYT comments on AI risk are either dismissive, or just laden with general anxiety about tech. (Indeed, the second-most recommended comment is deeply dismissive, and the third is generic anxiety/frustratio
My read of Bostrom's intent is that s-risks are deliberately excluded because they fall under the "arcane" category of considerations (per Evaluative Framework section), and this is supposed to be looking simply at Overton Window tradeoffs around lives saved.
However, I think you could still make a fair argument that s-risks could fall within the Overton Window if framed correctly, ex. "consider the possibility your ideological/political enemies win forever". This is already part of the considerations being made by AI labs and relevant governments in as simple terms as US vs. China.[1] Still, I think the narrower analysis done by Bostrom here is still interesting.
One might argue this is not a "real" s-risk, but ex. Anthropic's Dario Amodei seems pretty willing to risk the destruction of humanity over China reaching ASI first, according to his public statements, so I think it counts as a meaningful consideration in the public discourse outside of mere lives saved/lost.