A way I’ve been thinking about AI governance recently is in terms of different standard frames that strands of AI governance map onto. The current frame that’s most in vogue is treating AI as a standard product that carries substantial safety risks, and then applying standard policy tools from this area to regulate AI systems. One reason I find it useful is as a way of finding relevant case studies.
Some other frames that I think AI governance work falls within are:
A product that carries large negative externalities
This is the frame that’s being focused on the most at the moment, and I think motivates work like looking at how the aircraft and nuclear power industries achieve such low accident rates. I think lots of evals work also makes sense in this framework, insofar as evaluations aim to be the basis for standards regulators can adopt. I think some corporate governance work also falls into this category, like whistleblower protections. I think this frame makes sense when governments can unilaterally enact strong, competent regulations without being undone by national security concerns and regulatory arbitrage.
Innovation policy
One strand of AI governance work looks like innovation policy where the focus is on creating incentives for researchers to create new safety technologies. Lots of the responses to climate change fall within this frame, like subsidies for solar panels and funding for fundamental science aimed at technologies to reduce the harms of climate change. I think this type of policy is useful when coordination and regulation are really hard for some reason and so instead of adopting coordination measures (like the Kyoto or Paris climate agreements), it makes more sense to change the technology available to actors so it’s cheaper for them to produce fewer negative externalities. I think this approach works less well when, not only are actors not coordinating, but they’re competing with each other. Nuclear weapons probably look like this - it seems pretty hard to come up with innovations that change the incentives countries face so that they give up nuclear weapons of their own free will.
A national security risk
In this framework, AI is a potentially dangerous technology that adversarial state and or non-state actors can use this technology to pose criminal or terrorist threat. I think things like bio-evals are working in this framework, as is work focused on controlling open-source models, at least to some degree. In this frame, AI is treated like other WMD technologies like nuclear and chemical weapons, as well as other technologies that significantly enhance criminal capabilities like malware.
Preventing competitive dynamics
Work in compute governance aimed at preventing Chinese AI firms from accessing cutting-edge chips is in this category, as is work ensuring that AI firms have good information security. This work aims to generally prevent competition between actors that make it more expensive for them to adopt safety-enhancing technologies. This looks like a policy aimed at preventing race-to-the-bottom dynamics, for instance, a policy that tries to reduce the use of corporate tax havens.
As an instrument of great power conflict
This frames views AI as an important strategic technology that it’s important that - something like - the free world leads in. Some AI policy aimed at ensuring that the US has access to high-skilled immigration is in this camp. Other examples of this kind of policy are early nuclear weapons policy and early space technology.
Improving consumer welfare
A huge amount of normal policy work is aimed at improving consumer welfare. Some examples include competition policy, policy regulating consumer financial products, advertising restrictions, and arguably privacy policy. Quite a lot of non-catastrophic risk-focused AI policies might fall into this camp, like policies aimed at trying to ensure that LLM services have the same standards as relevant professional groups e.g., that when LLMs give financial advice it doesn’t carry the same risks that financial advice from individuals without financial planning licenses might give. Another example might be work trying to prevent chatbots from having the same quasi-addictive qualities that social media has.
Political economy
This is a very large category that includes work in bias and fairness camp, but also work around ensuring that there’s broadly shared prosperity following large-scale automation and ensuring election integrity. I’m grouping all of these categories because I think that they're focused on questions of the distribution of power and the economic consequences of this. Lots of the history of liberalism and the left has been focused on these kinds of questions, like the passage of the lords reform and the people’s budget under the British Liberal government of the very early 20th century, and LBJ’s great society and civil rates bills.
Potentially, work on AI sentience will be in this camp, structured similarly to anti-slavery work in Britain in the early 19th century, where the oppressed group wasn’t the primary locus for change (unlike say 2nd wave feminism.)
Military technology
In this frame, AI is a technology with the potential to make war worse and more dangerous. The campaign to stop killer robots fits in this category. Other examples of this kind of policy work include the banning of particularly nasty types of weapons like bouncing bombs, as well as more prosaic work like reducing the use of tactical nuclear weapons.
My impression is that for this kind of work to be effective it has to not put militaries that adopt these limitations at a severe disadvantage. I suspect this is part of the reason why laws of war that prevent targeting civilians have been reasonably effective - armies can still fight effectively while not, for instance, committing sexual violence, whereas they can fight effectively by using rubber bullets.
Innovation policy part 2 (electric boogaloo)
This frame is focused on the economic growth benefits of AI and focuses on questions like leveraging AI to improve healthcare delivery and drug discovery. I think this just looks like various standard innovation policy questions, like how to structure research grants to incentivize socially useful innovation, and loan programs for startups developing socially useful products.
CW: reference to sexual violence
A way I’ve been thinking about AI governance recently is in terms of different standard frames that strands of AI governance map onto. The current frame that’s most in vogue is treating AI as a standard product that carries substantial safety risks, and then applying standard policy tools from this area to regulate AI systems. One reason I find it useful is as a way of finding relevant case studies.
Some other frames that I think AI governance work falls within are:
A product that carries large negative externalities
This is the frame that’s being focused on the most at the moment, and I think motivates work like looking at how the aircraft and nuclear power industries achieve such low accident rates. I think lots of evals work also makes sense in this framework, insofar as evaluations aim to be the basis for standards regulators can adopt. I think some corporate governance work also falls into this category, like whistleblower protections. I think this frame makes sense when governments can unilaterally enact strong, competent regulations without being undone by national security concerns and regulatory arbitrage.
Innovation policy
One strand of AI governance work looks like innovation policy where the focus is on creating incentives for researchers to create new safety technologies. Lots of the responses to climate change fall within this frame, like subsidies for solar panels and funding for fundamental science aimed at technologies to reduce the harms of climate change. I think this type of policy is useful when coordination and regulation are really hard for some reason and so instead of adopting coordination measures (like the Kyoto or Paris climate agreements), it makes more sense to change the technology available to actors so it’s cheaper for them to produce fewer negative externalities. I think this approach works less well when, not only are actors not coordinating, but they’re competing with each other. Nuclear weapons probably look like this - it seems pretty hard to come up with innovations that change the incentives countries face so that they give up nuclear weapons of their own free will.
A national security risk
In this framework, AI is a potentially dangerous technology that adversarial state and or non-state actors can use this technology to pose criminal or terrorist threat. I think things like bio-evals are working in this framework, as is work focused on controlling open-source models, at least to some degree. In this frame, AI is treated like other WMD technologies like nuclear and chemical weapons, as well as other technologies that significantly enhance criminal capabilities like malware.
Preventing competitive dynamics
Work in compute governance aimed at preventing Chinese AI firms from accessing cutting-edge chips is in this category, as is work ensuring that AI firms have good information security. This work aims to generally prevent competition between actors that make it more expensive for them to adopt safety-enhancing technologies. This looks like a policy aimed at preventing race-to-the-bottom dynamics, for instance, a policy that tries to reduce the use of corporate tax havens.
As an instrument of great power conflict
This frames views AI as an important strategic technology that it’s important that - something like - the free world leads in. Some AI policy aimed at ensuring that the US has access to high-skilled immigration is in this camp. Other examples of this kind of policy are early nuclear weapons policy and early space technology.
Improving consumer welfare
A huge amount of normal policy work is aimed at improving consumer welfare. Some examples include competition policy, policy regulating consumer financial products, advertising restrictions, and arguably privacy policy. Quite a lot of non-catastrophic risk-focused AI policies might fall into this camp, like policies aimed at trying to ensure that LLM services have the same standards as relevant professional groups e.g., that when LLMs give financial advice it doesn’t carry the same risks that financial advice from individuals without financial planning licenses might give. Another example might be work trying to prevent chatbots from having the same quasi-addictive qualities that social media has.
Political economy
This is a very large category that includes work in bias and fairness camp, but also work around ensuring that there’s broadly shared prosperity following large-scale automation and ensuring election integrity. I’m grouping all of these categories because I think that they're focused on questions of the distribution of power and the economic consequences of this. Lots of the history of liberalism and the left has been focused on these kinds of questions, like the passage of the lords reform and the people’s budget under the British Liberal government of the very early 20th century, and LBJ’s great society and civil rates bills.
Potentially, work on AI sentience will be in this camp, structured similarly to anti-slavery work in Britain in the early 19th century, where the oppressed group wasn’t the primary locus for change (unlike say 2nd wave feminism.)
Military technology
In this frame, AI is a technology with the potential to make war worse and more dangerous. The campaign to stop killer robots fits in this category. Other examples of this kind of policy work include the banning of particularly nasty types of weapons like bouncing bombs, as well as more prosaic work like reducing the use of tactical nuclear weapons.
My impression is that for this kind of work to be effective it has to not put militaries that adopt these limitations at a severe disadvantage. I suspect this is part of the reason why laws of war that prevent targeting civilians have been reasonably effective - armies can still fight effectively while not, for instance, committing sexual violence, whereas they can fight effectively by using rubber bullets.
Innovation policy part 2 (electric boogaloo)
This frame is focused on the economic growth benefits of AI and focuses on questions like leveraging AI to improve healthcare delivery and drug discovery. I think this just looks like various standard innovation policy questions, like how to structure research grants to incentivize socially useful innovation, and loan programs for startups developing socially useful products.