Metacrisis as a Framework for AI Governance
The theory of metacrisis[1] put forward by Daniel Schmachtenberger and others has gained traction in some circles concerned with global risks[2]. But despite strong similarities in key claims, it remains unexplored within AI safety and governance discourse[3]. In this post I aim to bridge the gap by comparing related ideas across the two fields, focusing on how metacrisis theory can offer a coherent framework for AI governance research. Central to metacrisis theory is the idea that the main global risks facing humanity have common structural and cultural roots[4]. Catastrophic risks from AI, great power war and environmental damage, among others, are traced back to economic structures and dangerous system dynamics, as well as the cultural and psychological factors supporting them. There is significant overlap between this perspective and AI governance research. There are also areas where metacrisis theory can be seen as offering a new framing of research priorities - such as Schmachtenberger’s analysis of civilizational system dynamics and the need for a “third attractor”. Both metacrisis theory and AI governance research are broad fields covering multiple areas. Before diving into the details, I'll briefly situate each field relative to the other. Situating Metacrisis within AI Governance Research A fairly standard way of thinking about the range of research in AI governance is to distinguish the following four categories: * Strategy research: investigating conceptual foundations, forecasting likely AI risks and setting priorities for AI governance work * Industry-focused approaches: research on how to improve decisions at AI labs through corporate governance, and technical standards and evaluations * Government-focused approaches: research on possible executive and legislative action, including international negotiations and national policy * Field-building: developing the institutional infrastructure for, and raising awareness of AI governance as a d
Thanks for these thoughts. Yes this is broadly the direction I am headed, and planning to explore further in future posts. I agree that both repeated (IPD-style) interactions and kin-interactions are ways of developing the evolutionary prisoner's dilemma so as to 'integrate Moloch and the Goddess into a single model' as I put it at the end of my post. Another approach is to introduce spatial structure.
There are a number of different specific models to explore here, and I prefer to look at the simplest and clearest versions. Your presentation is very dense, and I think introduces some complications that are not necessary to make the point. For example I'm not sure... (read more)