This post is an adaptation of a prior post I made on my website. I'm interested to see what holes exist in the position that I'm missing. Epistemic Status: medium confidence.
Within the next 6 months, most AI startups will not be able to buy compute. The world’s compute is "sold out".
I put "sold out" in quotes, because in the current era, compute is rarely built speculatively. It's not like someone is sitting around with an idle cluster waiting for you to buy it. GPU clouds simply don't work like traditional clouds. In web servers, your customers are SaaS companies who are high-margin businesses. If the cloud raises prices on the SaaS company, the SaaS company isn't happy, but it doesn't go bankrupt. But if the GPU cloud raises prices on the AI lab, the lab may literally be unable to afford it, and thus they are truly price sensitive. When countless speculators deployed H100 clusters to sell on short term, on-demand contracts, customers did not suddenly decide to pay them the $6/hr their models assumed and a number of folks got wiped out.
So in every subsequent generation, when you rent a cluster from a cloud, you're always buying a net-new cluster. The cloud takes your contract, goes to the bank, and the bank writes a loan based on the quality of your credit. However, it's becoming increasingly difficult to get capacity online for folks who are not "investment grade".
Startups are not investment grade. So, to get capacity, they are asked to pay larger and larger upfront payments. We’ve seen it go as high as 100% of the contract paid up front, which was required because neither the lender nor the cloud could take on any additional credit risk.
Meanwhile, investment grade tenants are rare. The few investment grade tenants that remain are getting increasingly downrated as they take on more and more debt and then turn around to sell to buyers of tokens who themselves are not investment grade.
This is the credit bottleneck.
We expect the investment grade buyers of GPUs to increasingly ask that the buyers of tokens be either investment grade themselves or pay, up front, for a large deposit. Today, inference companies are able to serve at high margins, which means their cashflows may allow them to be seen as credit worthy even when their customers are not.
However, it's not clear that those margins are sustainable. As of writing, you can buy a B300 for $7/hr and then make about $30/hr selling inference on it. That’s a healthy margin, but there’s little to stop others from arbitraging that margin away. We expect GPU prices to rise, which means that margins will fall for the inference layer. Declining margins means it will become harder to secure a better rating, which means there’s a stronger incentive to secure investment grade customers for tokens.
When that happens, it becomes structurally harder to build a firm, in any industry. If you have credit, you get tokens, and if you get tokens, you build better products faster, make more money, and get better credit. Those tokens will become harder to acquire for customers with poor credit. Meanwhile, competition in the application layer will require you to spend more tokens faster; a Fortune 500 that can deploy $1b in tokens in a few days has an overwhelming advantage over a startup that first must raise $1b prior to their product existing.
The following are three scenarios for how things could play out, I'm curious if there are more.
Scenario one is that the frontier labs simply eat everything. Unlike open source inference, frontier labs have a larger claim to being able to retain their margin. Anthropic is generating significant margins & it's not crazy to assume OpenAI is as well. In this case, they may be able to fund new clusters with cashflow, plus their existing debt will get paid down with interest. However, unless you assume that AI demand slows down, the frontier labs will likely still attempt to exhaust their balance sheet & take debt. To avoid getting downrated, they will still need to prioritize investment grade token-buyers. As models improve, the ROI on token spend increases, which means their customers will want leverage. Today, "leverage" looks like a lab fronting an accounts payable risk to their customers via a net-30 billing or simply having a high limit with an SLA that lets you deploy a large burst spend quickly. That leverage will be extended to the Fortune 500, but likely not to the 2-person startup that may look suspicious from the perspective of the lender that the frontier lab must court in order to continue to scale.
Scenario two is that an investment grade counter party owns the assets and then either leases them to someone or backstops their lease to make them credit worthy. The simplest iteration of this is the Nvidia backstop that occurred for the frontier labs. Nvidia alone is likely unable to do this at scale, but they're not the only ones who have reasonable credit. With the right incentive or the right price, you could imagine a number of other businesses extending their credit worthiness to smaller firms. If you redesigned venture capital, it might look something like this.
Scenario three is that the collateral becomes more firm. Today, when you write a loan against the GPU hardware, it's hard to get a real price for the residual value of that hardware. Absent a firm number, you're forced to use the credit quality of the offtaker, continuing the credit crunch. Meanwhile, in the situation in which the offtaker fails, causing the cloud to fail, the lender may recoup hardware that they themselves cannot operate.
One comment I have heard when discussing this with folks is that this will simply blow over. But I suspect that these folks don't really think AI is real. If you agree with the sentence "AI will increase the rate of GDP growth" then you are implicitly agreeing that interest rates will rise and debt will become more expensive. When people think they're going to be richer in the future, they borrow to achieve that reality. More borrowing makes debt more expensive right up until people stop borrowing. Except if we really are at the beginning of it all, we may never stop borrowing.
I mean, most startups fail. I predict we won't run out of compute or credit, though prices may rise a fair bit, and that will cause some companies to fail, if they're mostly reselling tokens rather than actually adding value.
I think big companies are VERY good at risk management. They're using credit to build, and giving some credit to their consumers, but the terms are different - mostly long-term for building and short-term for using. Borrow long and sell short is generally fine. If some (or even most) customers stop paying, they get cut off and others take their place. Prices may rise a bit if bad debt becomes hugely significant. But I predict it won't change creditworthiness or investor confidence.
This post is an adaptation of a prior post I made on my website. I'm interested to see what holes exist in the position that I'm missing. Epistemic Status: medium confidence.
Within the next 6 months, most AI startups will not be able to buy compute. The world’s compute is "sold out".
I put "sold out" in quotes, because in the current era, compute is rarely built speculatively. It's not like someone is sitting around with an idle cluster waiting for you to buy it. GPU clouds simply don't work like traditional clouds. In web servers, your customers are SaaS companies who are high-margin businesses. If the cloud raises prices on the SaaS company, the SaaS company isn't happy, but it doesn't go bankrupt. But if the GPU cloud raises prices on the AI lab, the lab may literally be unable to afford it, and thus they are truly price sensitive. When countless speculators deployed H100 clusters to sell on short term, on-demand contracts, customers did not suddenly decide to pay them the $6/hr their models assumed and a number of folks got wiped out.
So in every subsequent generation, when you rent a cluster from a cloud, you're always buying a net-new cluster. The cloud takes your contract, goes to the bank, and the bank writes a loan based on the quality of your credit. However, it's becoming increasingly difficult to get capacity online for folks who are not "investment grade".
Startups are not investment grade. So, to get capacity, they are asked to pay larger and larger upfront payments. We’ve seen it go as high as 100% of the contract paid up front, which was required because neither the lender nor the cloud could take on any additional credit risk.
Meanwhile, investment grade tenants are rare. The few investment grade tenants that remain are getting increasingly downrated as they take on more and more debt and then turn around to sell to buyers of tokens who themselves are not investment grade.
This is the credit bottleneck.
We expect the investment grade buyers of GPUs to increasingly ask that the buyers of tokens be either investment grade themselves or pay, up front, for a large deposit. Today, inference companies are able to serve at high margins, which means their cashflows may allow them to be seen as credit worthy even when their customers are not.
However, it's not clear that those margins are sustainable. As of writing, you can buy a B300 for $7/hr and then make about $30/hr selling inference on it. That’s a healthy margin, but there’s little to stop others from arbitraging that margin away. We expect GPU prices to rise, which means that margins will fall for the inference layer. Declining margins means it will become harder to secure a better rating, which means there’s a stronger incentive to secure investment grade customers for tokens.
When that happens, it becomes structurally harder to build a firm, in any industry. If you have credit, you get tokens, and if you get tokens, you build better products faster, make more money, and get better credit. Those tokens will become harder to acquire for customers with poor credit. Meanwhile, competition in the application layer will require you to spend more tokens faster; a Fortune 500 that can deploy $1b in tokens in a few days has an overwhelming advantage over a startup that first must raise $1b prior to their product existing.
The following are three scenarios for how things could play out, I'm curious if there are more.
Scenario one is that the frontier labs simply eat everything. Unlike open source inference, frontier labs have a larger claim to being able to retain their margin. Anthropic is generating significant margins & it's not crazy to assume OpenAI is as well. In this case, they may be able to fund new clusters with cashflow, plus their existing debt will get paid down with interest. However, unless you assume that AI demand slows down, the frontier labs will likely still attempt to exhaust their balance sheet & take debt. To avoid getting downrated, they will still need to prioritize investment grade token-buyers. As models improve, the ROI on token spend increases, which means their customers will want leverage. Today, "leverage" looks like a lab fronting an accounts payable risk to their customers via a net-30 billing or simply having a high limit with an SLA that lets you deploy a large burst spend quickly. That leverage will be extended to the Fortune 500, but likely not to the 2-person startup that may look suspicious from the perspective of the lender that the frontier lab must court in order to continue to scale.
Scenario two is that an investment grade counter party owns the assets and then either leases them to someone or backstops their lease to make them credit worthy. The simplest iteration of this is the Nvidia backstop that occurred for the frontier labs. Nvidia alone is likely unable to do this at scale, but they're not the only ones who have reasonable credit. With the right incentive or the right price, you could imagine a number of other businesses extending their credit worthiness to smaller firms. If you redesigned venture capital, it might look something like this.
Scenario three is that the collateral becomes more firm. Today, when you write a loan against the GPU hardware, it's hard to get a real price for the residual value of that hardware. Absent a firm number, you're forced to use the credit quality of the offtaker, continuing the credit crunch. Meanwhile, in the situation in which the offtaker fails, causing the cloud to fail, the lender may recoup hardware that they themselves cannot operate.
One comment I have heard when discussing this with folks is that this will simply blow over. But I suspect that these folks don't really think AI is real. If you agree with the sentence "AI will increase the rate of GDP growth" then you are implicitly agreeing that interest rates will rise and debt will become more expensive. When people think they're going to be richer in the future, they borrow to achieve that reality. More borrowing makes debt more expensive right up until people stop borrowing. Except if we really are at the beginning of it all, we may never stop borrowing.