[LINK] Cochrane on Existential Risk

Salemicus

The finance professor John Cochrane recently posted an interesting blog post. The piece is about existential risk in the context of global warming, but it is really a discussion of existential risk generally; many of his points are highly relevant to AI risk.

If we [respond strongly to all low-probability threats], we spend 10 times GDP.

It's a interesting case of framing bias. If you worry only about climate, it seems sensible to pay a pretty stiff price to avoid a small uncertain catastrophe. But if you worry about small uncertain catastrophes, you spend all you have and more, and it's not clear that climate is the highest on the list...

All in all, I'm not convinced our political system is ready to do a very good job of prioritizing outsize expenditures on small ambiguous-probability events.

He also points out that the threat from global warming has a negative beta - i.e. higher future growth rates are likely to be associated with greater risk of global warming, but also the richer our descendants will be. This means both that they will be more able to cope with the threat, and that the damage is less important from a utilitarian point of view. Attempting to stop global warming therefore has positive beta, and therefore requires higher rates of return than simple time-discounting.

It strikes me that this argument applies equally to AI risk, as fruitful artificial intelligence research is likely to be associated with higher economic growth. Moreover:

The economic case for cutting carbon emissions now is that by paying a bit now, we will make our descendants better off in 100 years.

Once stated this way, carbon taxes are just an investment. But is investing in carbon reduction the most profitable way to transfer wealth to our descendants? Instead of spending say $1 trillion in carbon abatement costs, why don't we invest $1 trillion in stocks? If the 100 year rate of return on stocks is higher than the 100 year rate of return on carbon abatement -- likely -- they come out better off. With a gazillion dollars or so, they can rebuild Manhattan on higher ground. They can afford whatever carbon capture or geoengineering technology crops up to clean up our messes.

So should we close down MIRI and invest the funds in an index tracker?

The full post can be found here.

If we [respond strongly to all low-probability threats], we spend 10 times GDP.

It's a interesting case of framing bias. If you worry only about climate, it seems sensible to pay a pretty stiff price to avoid a small uncertain catastrophe. But if you worry about small uncertain catastrophes, you spend all you have and more, and it's not clear that climate is the highest on the list...

All in all, I'm not convinced our political system is ready to do a very good job of prioritizing outsize expenditures on small ambiguous-probability events.

It strikes me that this argument applies equally to AI risk, as fruitful artificial intelligence research is likely to be associated with higher economic growth. Moreover:

The economic case for cutting carbon emissions now is that by paying a bit now, we will make our descendants better off in 100 years.

Once stated this way, carbon taxes are just an investment. But is investing in carbon reduction the most profitable way to transfer wealth to our descendants? Instead of spending say $1 trillion in carbon abatement costs, why don't we invest $1 trillion in stocks? If the 100 year rate of return on stocks is higher than the 100 year rate of return on carbon abatement -- likely -- they come out better off. With a gazillion dollars or so, they can rebuild Manhattan on higher ground. They can afford whatever carbon capture or geoengineering technology crops up to clean up our messes.

So should we close down MIRI and invest the funds in an index tracker?

The full post can be found here.

Indeed. (Fun for commenters: come up with more. Asteroid impact. Banking system collapse. Massive crop failure from virus or bacteria. Antibiotic resistsance....) If we treat all threats this way, we spend 10 times GDP. It's a interesting case of framing bias. If you worry only about climate, it seems sensible to pay a pretty stiff price to avoid a small uncertain catastrophe. But if you worry about small uncertain catastrophes, you spend all you have and more, and it's not clear that climate is the highest on the list.

This seems like a very strange thing for an economics professor to say.

Suppose we make an isomorphic argument:

"Of course, one can buy insurance against, say, a car crash. Shouldn't we pay a bit more car insurance now, though the best guess is that it's not a worthwhile investment, as insurance against such tail risks? But the problem is, we could buy insurance against our house burning down, homeowner's insurance against being robbed, our iPod breaking, our husband dying (or our wife, or our children, or our pets), travel insurance about our upcoming vacation, insurance against losing our job, catastrophic health insurance, legal liability insurance, longevity insurance... (Commenters, have fun listing others.) But if we treat risks this way, we'll wind up spending 10 times our annual income on insuring against risks! It's an interesting case of framing bias: it may sound rational to insure against a house fire or a car crash or an income-earner dying unexpectedly, you spend all you have and more, so it's not clear that car crash insurance is highest on the list."

Doesn't sound quite so clever that time around, does it? But all I did was take the framework of his argument: "if one invests in insurance against X, then because there are also risks Y, Z, A, B, C, which are equally rational to invest in, one will also invest in risks Y...C, and if one invested against all those risks, one will wind up broke; any investment where one winds up broke is a bad investment; QED, investing in insurance against X is a bad investment." and substitute in different, uncontroversial, forms of insurance and risk.

What makes the difference? Why does his framing seem so different from my framing?

Well, it could be that the argument is fallacious in equating risks. Maybe banking system collapse has a different risk from crop collapse which has a different risk from asteroid impact, and really, we don't care about some of them and so we would not invest in those, leaving us not bankrupt but optimally insured. In which case his argument boils down to 'we should invest in climate change protection iff it's the investment which highest marginal returns', which is boringly obvious and not insightful at all because it means that all we need to discuss is the object-level discussion about where climate change belongs on "the list", and there's not a meta-level objection to investing against existential risks at all as the post is being presented as.

You are treating "investing in preventing X" as the same thing as "insuring against X." They are not the same thing. And they are doubly not the same thing on a society-wide level.

Insurance typically functions to distribute risk, not reduce it. If I get insurance against a house fire, my house is just as likely to burn down as it was before. However, the risk of a house fire is now shared between me and the insurance company. As Lumifer points out, trying to make your house fire-proof (or prevent any of the other risks you list) really ... (read more)

1Lumifer13y

The difference is that one set of risks is insurable and the other is not. An insurable risk is one which can be mitigated through diversification. You can insure your house against fire only because there are thousands of other people also insuring their houses against fire. One consequence is that insurance is cheaper than an individual guarantee: it would cost much more to make your specific house entirely fireproof. The other difference (and that one goes against Cochrane) is that normal insurable risks are survivable (and so you can assign certain economic value / utility / etc. to outcomes) while existential risks are not -- the value/utility of the bad outcome is negative infinity.

-1Dustin13y

(admittedly, I just skimmed the blog post, so I can be easily convinced my tentative position here is wrong) I'm not sure I see any difference between your proposed isomorphic argument and his argument. Assuming our level of certainty about risks we can insure against is the same as our level of (un)certainty about existential risks, and assuming the "spending 10 times our annual income" is accurate for both...the arguments sound exactly as "clever" as each other. I also am not sure I agree with the "boringly obvious and not insightful at all" part. Or rather, I agree that it should be boringly obvious, but given our current obsession with climate change, is it boringly obvious to most people? Or rather, I suppose, the real question is do most people need the question phrased to them in this way to see it? I guess what I'm saying is that it doesn't seem implausible to me is that if you asked a representative sample of people if climate change protection was important to invest in they would say yes and vote for that. And then if you made the boringly obvious argument about determining where it belongs on the list of important things, they'd also say yes and vote for that.