Racing Towards a New Technology Is a Collective Choice, Not an Inevitable Consequence of Incentives

 

Before I started thinking about AI policy, I was working in the trying-to-get-fusion industry.

There are some significant similarities between AI and fusion.

  • Both are emerging technologies.
  • Both have the potential to have significant effects on the global economy.
  • Both are dominated by O(10) major actors, which include both private companies and governments.
  • Both currently require a capital expenditure of O($1 billion) to make something impressive.
  • Both of the leading companies are about the same size, adjusting for when the companies were founded.
  • Both feel futuristic.

The incentive structures for the two industries feel similar.

However, the culture of these two industries feels very different. Fusion companies are not racing in the way that AI companies are. People who are trying to build a fusion power plant are trying to sell electricity to the grid as soon as possible, but they approach this endeavor with a cooperative attitude, rather than a competitive one.

This culture of collaboration has deep roots in the fusion community. After some surprisingly good results on the T-3 tokamak in Moscow, in 1969, Soviet scientists invited British scientists to come measure the temperature on the experiment themselves. From then on, fusion researchers refused to be part of the Cold War and collaborated extensively across the Iron Curtain. Even the height of tensions in the 1980s saw the seed of a flagship international fusion experiment: ITER. ITER is the largest scientific collaboration ever undertaken, with member states representing over half of the world’s population.[1]

ITER is less of a flagship now than it once was, due to the rise of private companies with plausible paths to fusion.[2]

I will divide private fusion companies into two categories: academic fusion companies and non-academic fusion companies. Academic fusion companies were founded by people who have extensive experience in academic or national lab fusion efforts, use designs that have been researched for decades,[3] attend academic conferences, and publish their results following the norms of peer-reviewed science. Examples of academic fusion companies include Commonwealth Fusion Systems (CFS; their first experiment is called SPARC) and Type One Energy.[4] Non-academic fusion companies were founded by people outside of the fusion community, use novel designs, may or may not attend academic plasma physics conferences, and announce their results following the norms of industries with trade secrets. I won’t name names here because I think most of them are unserious, and I don’t want to besmirch people who are trying to do cool things, even if I think they are not going to succeed. Helion is frustratingly on the boundary, and I’ll talk about them later. This post is mostly about academic fusion companies.

Academic fusion companies inherited the culture of the existing fusion community at universities and national labs. They cooperate more than they compete. The attitude is that everyone will be happy if anyone succeeds.

Most of my evidence comes from interacting with members of the community. For a bit more legible evidence, here are some quotes from a Fusion Industry Association panel:[5]

  • Francesca Ferrazza stated that fusion “needs to have a global effort” and that “competition and cooperation go together.”
  • Thomas Forner: “I don’t think any single company [alone] will be able to solve fusion;” it “will take collaborative work.”
  • Moritz von der Linden: We “cannot just bet on a horse, we need to bet on the race.”
  • Dr. Mark Pleško underlined the importance of “cooperation between government and industry” to accelerate development.

We can also look at whether there is more substantial cooperation.

In 2018, there was a big discovery that caused a dramatic increase in the number of academic fusion companies: the development of high temperature superconducting 'tape'. While high temperature superconductors have been known for several decades, most of them were specks of brittle material on a special substrate. MIT materials scientists were able to create a high temperature superconductor on a metal backing that can be made large and robust enough to use in construction. At liquid nitrogen temperature, this allows for twice as strong of magnetic fields as were planned for ITER, which reduces the necessary volume (and cost) of a reactor by a factor of 8. This put first generation fusion reactors within reach of venture capital. The MIT group which first planned a fusion reactor using this high temperature superconductor is now a startup: Commonwealth Fusion Systems. I think they are the organization closest to building a fusion power plant.

Commonwealth Fusion Systems has also made this material available to other fusion companies. Most of the academic fusion companies are planning to use this high temperature superconducting tape. CFS isn’t even the first company to use it in an experiment: that honor goes to the University of Wisconsin & Realta Fusion.[6][7]  Unconfirmed rumor has it that CFS is selling this tape at or even below cost of production, in exchange for equity in other fusion startups. Even if CFS’s design for a fusion reactor fails (and I don’t think it will), they have made themselves a central part of the fusion supply chain.[8] 

The most important new material for fusion reactors is not being closely guarded by its inventors. It is being made readily available for new competing companies to use.

This is not the only example of substantive collaboration. Simulations and diagnostic equipment are also extensively shared. Academic fusion companies, university groups, and national labs are mutually supporting.

Someone might expect that cooperative systems are unstable to defectors. If one organization receives all of the benefits of other companies’ cooperation, but does not share their discoveries, they could gain an advantage. The fusion industry does have one potentially serious competitor that is less involved in the cooperative system: Helion. It would be unfair to call Helion a defector – they are older than the academic fusion companies – but they do follow different norms.

I said above that Helion is sort of an academic fusion company. Their founders have a background in plasma propulsion for spacecraft, which is academic plasma physics, but not quite academic fusion.[9] They are not using a design that has been researched for decades, but they have pursued a serious experimental program to catch up. Helion recently turned on their 7th generation experiment, Polaris. They do attend conferences, and sometimes publish peer-reviewed results, but less so for their last two experiments.[10] Helion has been making big promises, and might actually fulfill some of them,[11] but a lot of the academic fusion community is still skeptical. Helion is also more comfortable talking about "leading the race."[12]

Helion’s distinct approach, both technically and socially, has not disrupted the cooperative nature of the academic fusion companies. Maybe this will change if Helion proves to be successful sooner than CFS. I doubt it. Partially because I am uncertain if Helion will be successful, but also because I expect academic fusion to be more effective at marshaling talent and (less confidently) capital over the long run. I expect that 50 years from now, barring any dramatic changes to society, the descendants of today’s academic fusion companies will be producing more electricity than the descendants of Helion.

When domestic competition is insufficient to justify racing, someone could also point towards China. China has been putting significant resources into fusion. One of the best current fusion experiments, EAST, opened in China in 2006. They are also pushing ahead with next generation experiments. Unfortunately, they are not as public with their plans as the other players, and it looks like their plans are changing. I tried to figure out what’s going on, but am still confused.[13] I do not know if they cooperate with Western fusion startups, but China is definitely part of the international academic & governmental fusion community. They are part of ITER, and have extensive collaborations between scientists at EAST and other tokamaks around the world. The fusion community included the Soviet Union at the height of the Cold War, and it continues to include China today.

I have sometimes wondered if the cooperative nature of the fusion community has been the most effective strategy, especially since fusion progress has been constrained by scarcity of government research funding. Going to the moon is exciting, but beating the Russians to the moon gets the Budget Office’s attention.[14] For an even more specific example, I wonder what would have happened if, in 2017, someone had tweeted at the president:

CHINA is winning the Race to Fusion, 
but WE CAN WIN with TRUMP TOKAMAK.

This moment has passed. The best way to support fusion today would probably not be for the US government to build a big new tokamak. The institutions of the fusion community have changed with the rise of the academic fusion companies, but the cooperative culture persists.

The AI community has very different social dynamics than the fusion community. I don’t want to go into too much detail here because this does not feel like my value add to this conversation – there are a lot of people here who are following the leading AI labs more closely than I am.

The leading AI labs openly talk about racing and beating their competitors. Companies do not share their key technologies, encouraging other people to make startups using their technology to try to achieve similar goals. Some large institutional actors are less explicit about racing and invest in multiple AI companies. These actors are also less likely to believe in the promise of future AI. Nvidia perhaps plays a similar role to CFS in being an essential part of other companies’ supply chain. However, Nvidia was founded as a chip company, not as a trying-to-get-AGI company that chose to make its chips available to its competitors. There is also the open source community, which feels like a separate culture even though the industry overlaps. When a major lab chooses to make their model open source, they do not do so because they want to give their competitors tools to make AGI. I do not have as good of a feel for the AI community as I do for the fusion community, but it is clear that the two cultures are very different.

I do not think that the reason for the difference is that the incentive structures are different. As I said at the beginning, the two industries have fairly similar structures. The benefits AI companies believe they will receive if they succeed are larger than the benefits fusion companies believe they will receive if they are successful – although both are substantial.[15] CFS might be more dominant in its industry than OpenAI, but Helion claims they are ahead and ITER has an over 10 times larger budget. All industries are different, but I think that these are close enough to expect similar race dynamics if the industries were merely reflections of the underlying incentives.

I think that the difference is because the industries have made different collective choices. The fusion community has chosen to create a cooperative culture, while the AI community has chosen to create a competitive culture. To understand how this has happened, we could look at specific choices made by leaders of the community at key junctures. One example is the choice of the Culham Five to visit the T-3 tokamak in the Soviet Union, or of Lev Artsimovitch to invite them.[16] Another example is the choice by the founders of Commonwealth Fusion Systems to make their high temperature superconducting tape available to others. These particular instances do not tell the entire story. They are heavily influenced by, and help create, expectations and norms that pervade the community. These expectations determine what leaders are more likely to do by default and what employees are more likely to demand. Breaking these expectations would lower their respect in the community, making it harder to attract talented new hires and collaborations.

On the other hand, if the community’s norms and expectations involve trying to win a race against your competitors, the behaviors of actors in the industry will be very different. This collective choice can be more powerful than institution design, as was dramatically demonstrated by the attempted ouster of Sam Altman from OpenAI. Once such a culture is created, it is difficult to change. Changing the culture involves the hard work of persuading a large number of people, and convincing leaders to take different actions.

People and companies will race when they collectively believe that they are in a race. This is not merely a consequence of the incentives facing the industry. Different industries with similar incentives can have different race dynamics if their people collectively choose to embrace different cultures.

  1. ^

    The member states of ITER are Europe (EU + UK + Switzerland), Russia, India, China, South Korea, Japan, and the United States.

  2. ^

    For a longer discussion of the new fusion companies, see my book review of The Future of Fusion Energy. It is a few years old now, but I still endorse the conclusions and most of the predictions.

  3. ^

    I have heard people who are not in plasma physics use this as an argument against academic fusion companies. If e.g. tokamaks have been researched for decades and have not proven successful, isn’t this evidence that tokamaks are not the correct approach? This is sort of true: existing tokamaks will not make a functioning power plant. You need significantly larger scale, either in size (ITER) or magnetic field (SPARC). Tokamaks at this scale have not been tested. We can more easily use what we’ve learned on smaller tokamaks if we pursue this strategy than if we pursue a completely new design.

  4. ^

    A more complete list is: 

    The number of these companies has doubled since I wrote my book review 2.5 years ago.

    General Atomics (US, tokamak) should maybe also be on the list, but they’re maintaining an existing experiment rather than trying to build a power plant.

  5. ^

    The global fusion industry in 2022: Report launch event in Brussels. Fusion Industry Association. (2022) https://www.fusionindustryassociation.org/the-global-fusion-industry-in-2022-report-launch-event-in-brussels/.

  6. ^

    Realta Fusion is a spinoff of the University of Wisconsin’s plasma physics group. This experiment is substantially smaller than CFS’s first experiment, so they are still farther away from getting fusion.

  7. ^

    Commonwealth Fusion Systems Delivers Superconducting Magnets to University of Wisconsin's WHAM Project. Commonwealth Fusion Systems. (2024) https://cfs.energy/news-and-media/commonwealth-fusion-systems-delivers-hts-magnets-to-uw-wham-project.

  8. ^

    I asked someone from CFS at a conference what they would do if a stellarator company ends up working better than their tokamak. His response was: “We’d probably buy them.”

  9. ^

    Human subcultures are nested fractally. https://xkcd.com/1095/.

  10. ^

    Sorry David Kirtley, but tweeting an emoji of a lightbulb is not up to the standards of scientific publishing.

  11. ^

    Microsoft expects that a 50 MW fusion power plant will come online by 2028. Helion also promised that, in 2024, its seventh generation experiment would turn on (which it did) and “demonstrate the ability to produce electricity.” They haven’t said anything yet about the second part of that prediction.

    Helion announces world’s first fusion energy purchase agreement with Microsoft. Helion. (2023) https://www.helionenergy.com/articles/helion-announces-worlds-first-fusion-ppa-with-microsoft/.

  12. ^

    Helion Announces $425M Series F Investment to Scale Commercialized Fusion Power. Helion. (2025) https://www.helionenergy.com/articles/helion-announces-425m-series-f-investment-to-scale-commercialized-fusion-power/.

    Note that this quote is from one their investors, rather than someone at the company itself. It is something that they choose to include in a press release.

  13. ^

    Here is what I have figured out about China’s fusion program:

    Starting around 2010, China began designing a large tokamak experiment called CFETR. It is intended to be a complement and successor to ITER, and should be finished sometime in the 2030s. 

    Toward the end of 2021, China noticed that Commonwealth Fusion Systems seemed to be making substantial progress, and announced a new smaller, higher field tokamak, named BEST, to be completed in 2027. It is not clear where they plan on getting their magnets from. I haven’t heard of any agreement with CFS. There is a Chinese startup, Energy Singularity, that is planning on building high temperature superconducting tokamaks, but their current magnets are an order of magnitude too weak. My guess is that China does not currently have a source of strong enough magnets to build BEST. 

    Also, there are satellite images of a big new laser fusion facility which they have not announced. This is probably mostly focused on nuclear stockpile management (like NIF), rather than working towards a power plant.

  14. ^

    This sentence is intentionally vague as to whether funding fusion (or the Space Race) at this scale would have been a good use of government resources. The point I am trying to make here is that we would have made progress faster with more money.

  15. ^

    The believed benefits are roughly: save the Earth from climate change (and make lots of money) vs. become ruler over a substantial portion of the observable universe (and make lots of money). The difference here might help explain why fusion attracts idealists and AI attracts megalomaniacs.

    I do not think this is a complete explanation. Neither of these potential futures is empirically established. Both are human-created narratives.

    Also, ‘Fusioneers are idealists and artificial intelligenciers are megalomaniacs’ is kind of my point.

  16. ^

    Robert Arnoux. Off to Russia with a thermometer. ITER. (2009) https://www.iter.org/node/20687/russia-thermometer.

New Comment
11 comments, sorted by Click to highlight new comments since:
[-]mishka166

In the AI community, the transition from the prevailing spirit of cooperation to a very competitive situation happened around the GPT-3 revolution. GPT-3 brought unexpected progress in the few-shot learning and in program synthesis, and that was the moment when it became clear to many people that AI was working, that its goals were technologically achievable, and many players in the industry started to estimate time horizons as being rather short.

Fusion has not reached its GPT-3 moment yet; that's one key difference. Helion has signed a contract selling some of its future energy to Microsoft, but we have no idea if they manage to actually deliver (on time, or ever).

Another key difference is, of course, that strong AI systems are expected to play larger and larger role in making future AIs.

In fusion this "recursion" is unlikely; the energy needed to make more fusion stations or to create new fusion designs can come from any source...

I think that I predict the opposite (conditional on what exactly is being predicted).

What exactly would count as a GPT-3 moment for fusion? How about an experiment demonstrating reactor-like conditions? This is roughly equivalent to what I referred to as 'getting fusion' in my book review.

My prediction is that, after Commonwealth Fusion Systems gets Q > 5 on SPARC, they will continue to supply or plan to supply HTS tape to at least 3 other fusion startups.

I'd say that the ability to produce more energy overall than what is being spend on the whole cycle would count as a "GPT-3 moment". No price constraints, so it does not need to reach the level of "economically feasible", but it should stop being "net negative" energy-wise (when one honestly counts all energy inputs needed to make it work).

I, of course, don't know how to translate Q into this. GPT-4o tells me that it thinks that Q=10 is what is approximately needed for that (for "Engineering Break-even (reactor-level energy balance)"), at least for some of the designs, and Q in the neighborhood of 20-30 is what's needed for economic viability, but I don't really know if these are good estimates.

But assuming that these estimates are good, Q passing 10 would count as the GPT-3 moment.

What happens then might depend on the economic forecast (what's the demand for energy, what are expected profits, and so on). If they only expect to make profits typical for public utilities, and the whole thing is still heavily oriented towards publicly regulated setups, I would expect continuing collaboration.

If they expect some kind of super-profits, with market share being really important and with expectations of chunks of it being really lucrative, then I would not bet on continuing collaboration too much...

I would also make the same prediction for Q > 10. Or when CFS first sells electricity to the grid. These will be farther into the future, but I do not think that this culture will have changed by then.

I think a crucial distinction, which you touch on but perhaps don't fully emphasize, lies in the downstream consequences of success in each field. While both are transformative, the nature of that transformation is radically different.

Fusion, if achieved at scale, primarily addresses the energy sector. It's a substitutional technology. It replaces fossil fuels with a cleaner, more abundant alternative, mitigating climate change and potentially altering geopolitical power dynamics related to energy resources. This is undeniably significant. However, beyond the energy sector and related geopolitical shifts, its direct impact on other aspects of the economy and society is likely to be relatively contained. It doesn't fundamentally rewrite how most things are done. We still need doctors, teachers, farmers, artists, etc., and their jobs, while perhaps indirectly affected by cheaper energy, are not fundamentally changed in character.

AI, particularly AGI, is fundamentally different. It's not merely substitutional; it's augmentative and potentially autonomous across a vast range of cognitive tasks. This has the potential to reshape virtually every human activity, from scientific research (including, ironically, accelerating fusion research, as you point out) to art, governance, and even warfare. The breadth and depth of potential change are orders of magnitude greater than with fusion.

Furthermore, AI exhibits a strong tendency towards winner-take-all (or winner-take-most) dynamics.The recursive self-improvement potential of a sufficiently advanced AI creates a powerful feedback loop. Once an entity achieves a certain threshold of general intelligence, it can potentially improve itself at an accelerating rate, making it exceedingly difficult for others to catch up. This "foom" potential, however unlikely some may deem it, creates a qualitatively different strategic landscape compared to fusion. The order of arrival matters immensely in AI in a way it simply doesn't in fusion. It's not just about achieving AGI; it's about who achieves it first and what safeguards are in place before that happens.

While both fusion and AI are important technological pursuits, the stakes and strategic implications are vastly different. The cooperative model described in the fusion community might be well-suited to its particular landscape. However, given the potential for rapid, self-driven escalation and the winner-take-all dynamics of AGI, a purely cooperative approach in AI seems, at best, strategically naive and, at worst, existentially risky. The incentive structures are, in fact, very different because of the outcomes, even if the initial industry structures appear superficially similar. Even if there is collaboration the incentives are heavily tilted towards defection as it favours them immensely.

I agree that this is plausibly a real important difference, but I do not think that it is obvious.

The most recent augmentative technological change was the industrial revolution. It has reshaped virtually every every activity. It allowed for the majority of the population to not work in agriculture for the first time since the agricultural revolution.

The industrial revolution centered on energy. Having much cheaper, much more abundant energy allowed humans to use that energy for all sorts of things. 

If fusion ends up being similar in cost to existing electricity production, it will be a substitutional technology. This is the thing that we are working on now (well, also making it work at all). People who work in fusion focus on this because it is the reasonable near/medium term projection. If fusion ends up being substantially cheaper, it will be an augmentative technology. It is not at all clear that this will happen, because we can't know how the costs will change between the first and thousandth fusion power plant.

Notably, we don't know if foom is going to be a thing either.

The narrative around the technology is at least as important as what has happened in the technology itself. The fusion community could frequently talk about how incredible the industrial revolution was, and how it powered Britain to global dominance for two centuries. A new source of energy might do the same thing ! But this is more hype than we feel we ought to offer, and the community's goal is not to create a dominant superpower.

Even if foom is going to happen, things would look very different if the leaders credibly committed to helping others foom if they are first. I don't know if this would be better or worse from a existential risk perspective, but it would change the nature of the race a lot.

we can't know how the costs will change between the first and thousandth fusion power plant.

Fusion plants are manufactured. By default, our assumption should be that plant costs follow typical experience curve behavior. Most technologies involving production of physical goods do. Whatever the learning rate x for fusion turns out to be, the 1000th plant will likely cost close to x^10. Obviously the details depend on other factors, but this should be the default starting assumption. Yes, the eventual impact assumption should be significant societal and technological transformation by cheaper and more abundant electricity. The scale for that transformation is measured in decades, and there are humans designing and permitting and building and operating each and every one, on human timescales. There's no winner take all dynamic even if your leading competitor builds their first commercial plant five years before you do.

Also: We do have other credible paths that can also greatly increase access to comparably low-cost dispatchable clean power on a similar timescale of development, if we don't get fusion.

we don't know if foom is going to be a thing

Also true, which means the default assumption without it is that the scaling behavior looks like the scaling behavior for other successful software innovations. In software, the development costs are high and then the unit costs in deployment quickly fall to near zero. As long as AI benefits from collecting user data to improve training (which should still be true in many non-foom scenarios) then we might expect network effect scaling behavior where the first to really capture a market niche becomes almost uncatchable, like Meta and Google and Amazon. Or where downstream app layers are built on software functionality, switching costs become very high and you get a substantial amount of lock-in, like with Apple and Microsoft.

Even if foom is going to happen, things would look very different if the leaders credibly committed to helping others foom if they are first. I don't know if this would be better or worse from a existential risk perspective, but it would change the nature of the race a lot.

Agreed. But, if any of the leading labs could credibly state what kinds of things they would or wouldn't be able to do in a foom scenario, let alone credibly precommit to what they would actually do, I would feel a whole lot better and safer about the possibility. Instead the leaders can't even precommit credibly to their own stated policies, in the absence of foom, and also don't have anywhere near a credible plan for managing foom if it happens.

Jeffrey, I appreciate your points about fusion's potential, and the uncertainty around "foom." However, I think framing this in terms of bottlenecks clarifies the core difference. The Industrial Revolution was transformative because it overcame the energy bottleneck. Today, while clean energy is vital, many transformative advancements are primarily bottlenecked by intelligence, not energy. Fusion addresses an important, existing constraint, but it's a step removed from the frontier of capability. AI, particularly AGI, directly targets that intelligence bottleneck, potentially unlocking progress across virtually every domain limited by human cognitive capacity. This difference in which bottleneck is addressed makes the potential transformative impact, and thus the strategic landscape, fundamentally distinct. Even drastic cost reductions in energy don't address the core limiting factor for progress in areas fundamentally constrained by our cognitive and analytical abilities.

This seems false given that AI training will be/is bottlenecked on energy.

Great, thought-provoking post. The AI research community certainly felt much more cooperative before it got an injection of startup/monopoly/winner-take-all thinking. Google Brain publishing the Transformer paper being a great example.

I wonder how much this truly is narrative, as opposed to AI being genuinely more winner-take-all than fusion in the economic sense. Certainly the hardware layer has proven quite winner-take-all so far with NVDA taking a huge fraction of the profit; same with adtech, the most profitable application of (last-generation) AI, where network effects and first mover advantages have led to the dominance of a couple of companies. 

Global foundation model development efforts being pooled into an international consortium like ITER or CERN seems quite good to me in terms of defusing race dynamics. Perhaps we will get there in a few years if private capital loses interest in funding 100B+ training runs.

[-]Ben20

Something related that I find interesting, for people inside a company, the real rival isn't another company doing the same thing, but people in your own company doing a different thing.

Imagine you work at Microsoft in the AI research team in 2021. Management want to cut R&D spending, so either your lot or the team doing quantum computer research are going to be redundant soon. Then, the timeline splits. In one universe, Open AI release Chat GPT, in the other PsiQuantum do something super impressive with quantum stuff. In which of those universes do the Microsoft AI team do well? In one, promotions and raises, in the other, redundancy.

People recognise this instinctively. Changing companies is much faster and easier than changing specialities. So people care primarily about their speciality doing well, their own specific company is a secondary concern.

A fusion expert can expert at a different fusion company way faster and more easily than they can  become an expert in wind turbines. Therefore, to the fusion expert all fusion companies are on the same side against the abominable wind farmers. I suspect this is also true of most people in AI, although maybe when talking to the press they will be honour bound to claim otherwise.

I wonder if any of the perceived difference between fusion and AI might be which information sources are available to you. It sounds like you have met the fusion people, and read their trade magazines, and are comparing that to what mainstream news says about AI companies (which won't necessarily reflect the opinions of a median AI researcher.).

Curated and popular this week