All of Kieran Farr's Comments + Replies

Thanks author for this amazing piece, especially bibliography to followup up on some of the direct sources. While I had been introduced to Normal Accident Theory a decade ago, I wasn't familiar with the HEVCR model and it definitely opens up new -- and very practical -- doors to thinking about existential risk in the context of AI.

If use the HEVCR what are the actual "exposure" and "vulnerability" issues for humanity? At least, I would assert that the exposure and vulnerability will vary widely between individuals and population groups. That is a stunning insight that in and of itself deserves future work.

3pendingsurvival
Thanks! I think the broader lesson to be drawn from this is that the EA/LessWrong/x-risk nexus really needs to make an effort to seek out and listen to people who have been working on related things for decades; they have some really useful things to say! There seems to be a pervasive tendency here to try to reinvent the wheel from first principles, or else exclusively rely on a handful of approaches that fit the community's highly quantitative/theoretical/high-modernist sensibilities---decision theory, game theory, prediction markets, judgemental forecasting, toy modelling, etc.---at the expense of all others. Both of these approaches are rarely productive in my view. (I am aware that this is not a new argument, but it bears making). Some organisations are much scientifically omnivorous/open to multiple perspectives and approaches; I'm a big fan of both CSER and the GCRI, for instance.[1] The Boring Apocalypses paper sketches out some ideas about existential exposures and vulnerabilities, but it's very much opening the conversation rather than offering the final word. You've definitely identified a major research gap, one made shocking by the fact that a clear majority of disaster risk reduction work is focused on reducing vulnerabilities and exposures rather than hazards. The point on variability of vulnerability and exposure is also well-taken. Vulnerability usually increases along all the lines you would expect: people living in poverty, marginalised minorities, etc. are typically much more at risk (c.f. the Pressure and Release model). 1. ^ Disclosure: I have worked at CSER as a Visiting Researcher.

Maybe it's even worse and we're just ms dos. Nobody bothers to emulate us except for the fun games