This post is crossposted from my Substack. Original here.

A modified version of this essay is now part of a much more comprehensive essay series, The Intelligence Curse.

Edited to add: The main takeaway of this post is meant to be: Labour-replacing AI will shift the relative importance of human v non-human factors of production, which reduces the incentives for society to care about humans while making existing powers more effective and entrenched. Many people are reading this post in a way where either (a) "capital" means just "money" (rather than also including physical capital like factories and data centres), or (b) the main concern is human-human inequality (rather than broader societal concerns about humanity's collective position, the potential for social change, and human agency).

I've heard many people say something like "money won't matter post-AGI". This has always struck me as odd, and as most likely completely incorrect.

First: labour means human mental and physical effort that produces something of value. Capital goods are things like factories, data centres, and software—things humans have built that are used in the production of goods and services. I'll use "capital" to refer to both the stock of capital goods and to the money that can pay for them. I'll say "money" when I want to exclude capital goods.

The key economic effect of AI is that it makes capital a more and more general substitute for labour. There's less need to pay humans for their time to perform work, because you can replace that with capital (e.g. data centres running software replaces a human doing mental labour).

I will walk through consequences of this, and end up concluding that labour-replacing AI means:

The ability to buy results in the real world will dramatically go up
Human ability to wield power in the real world will dramatically go down (at least without money); including because:
1. there will be no more incentive for states, companies, or other institutions to care about humans
2. it will be harder for humans to achieve outlier outcomes relative to their starting resources
Radical equalising measures are unlikely

Overall, this points to a neglected downside of transformative AI: that society might become permanently static, and that current power imbalances might be amplified and then turned immutable.

Given sufficiently strong AI, this is not a risk about insufficient material com...

wahala

wahala

wahala

wahala