Ask how AI safety reached this point, and most people give you a neat story. A small group of philosophers raised the first concerns about superintelligence. The arrival of large language models changed that, and many researchers turned to the alignment problem. The story is too tidy, and it leaves out the interesting part. Examine timelines, contributors, funding sources and publications, and it falls apart. New directions appear. A few merge into the mainstream, others fade out, and money and research attention rarely align; one tends to follow the other by years. I wanted to study that pattern rather than declare it, so I built the entire project from the ground up.
Below is a table of 323 documented events, 129 actors and 18 directions. Its timeline runs from 2005 through June 2026, and the final year covers only six months. Next to it, a separate set of arXiv attention proxies covers 23 of those directions. I also built a deduplicated safety corpus from a single combined query, so no paper is counted twice. Each row is one verifiable fact backed by a primary source, tagged with year, actor, direction, type, amount, source, and my confidence. Every chart below draws on that table, and the path was never straight. Sort the events by year, funding and attention, and you see fresh strands open up, a handful fold into the mainstream, several fall quiet, while funding and research stay out of step. The charts below are all generated from this base.
Main takeaways
Two kinds of findings came out of the data. Some just put numbers on what the field already half-knew. Others genuinely surprised me, so those come first.
Surprises:
Money and attention are systematically out of sync. Interpretability's scientific attention runs about 37 times ahead of what its roughly $1 million in grants would predict, while governance money ran ahead of its publications. Funding and research almost never arrive together.
Government money quietly overtook philanthropy around 2023. Half a dozen national institutes (UK AISI, ARIA, Canada, Australia, NSF, EU) turned a one-donor field into an international, increasingly governmental one, and about $268 million of venture equity poured into safety startups, money that expects a return, not a grant.
"Consolidation" is really proliferation. Even as attention and money concentrate around a few names, 2024 became a record year for founding new organisations (24 of them), a wave of one-to-five-person specialist shops in interpretability, AI security and chain-of-thought monitoring.
The word "Safety" is quietly leaving the field's names. In 2025 the UK AI Safety Institute became the AI Security Institute, the US AI Safety Institute became CAISI, and Open Philanthropy, the donor that started it all, renamed itself Coefficient Giving.
Going quiet is not the same as dying. Reward modeling's last discrete event is 2017, yet it runs to 2025 under the RLHF banner. Directions usually get absorbed or slip into the background, they rarely just die.
Expected, now with numbers behind it:
The center of gravity moved from philosophy to empirical work. Foundations-and-strategy events fall from 67% of the field in 2005-2013 to 6% by 2024-2026, while technical safety climbs to about 60%, concentrating around a handful of players, with Anthropic the single most-connected node.
RLHF won and dissolved into mainstream ML. Its arXiv proxy runs from 25 papers in 2021 to 1777 in 2025, precisely because it stopped being a safety topic and became standard practice.
Governance exploded after ChatGPT. Money on the governance subtrack rose from $0.4 million in 2018 to $18.4 million in 2023, roughly a 46-fold jump in five years.
Each dot is an event. The organisation axis is sorted by year of appearance, so the dots form a diagonal from the bottom-left up to the top-right. The track colour is a chronological gradient by birth year, so the field's colour drifts over time at a glance. The bottom panel stacks the events by track per year in the same colours, and the activity shifts from early macrostrategy and agent foundations toward later interpretability, evaluations and AI control. One caveat: that panel counts events, so it shows collection density rather than real-world activity. The eras are hatched and stars mark the milestone papers. (interactive).
The timeline is sorted by the start of a track, but it doesn't reveal who survived to today. For that there's a separate lifespan chart: a line from birth to the last recorded event, tracks ordered by birth year. One note here: a missing late event is not a dead track (again collection density, per the caveat above). So the chart carries an independent signal, a diamond on the last year a track still publishes on arXiv, wherever a curve exists. A track can fall silent in the event log and still be alive.
The line runs from a direction's first recorded event to its last, and the diamond marks the last year it still publishes on arXiv, an independent proxy across 23 directions. The two often diverge. Reward modeling's last event is 2017, yet its publications, now under the RLHF banner, run to 2025. Value learning went quiet as discrete events by 2021, and agent foundations thins to a lone 2024 marker with MIRI's pivot. Both stay alive on arXiv. So a track rarely just dies. More often it gets absorbed, or slips into the background. There is more on that in the Cross-cutting patterns chapter, after the eras. (interactive).
Here is another look at tracks over time, now counting the number of events in the base per year:
The stacked area shows how many events of each direction landed in the base each year. A reminder of caveat 2: the height is collection density, not real work volume, and the late years (2026 is only the first half) are hatched as incomplete. So read the shape and the order of appearance rather than the absolute values. The field sits almost empty until 2012 and 2013. It then explodes in 2015 to 2017 with the first technical tracks. And from 2021 to 2024 it keeps growing as new directions such as evaluations, AI control and model organisms pile on top of the older ones. (interactive).
One more overview map pulls the whole route together, tracing how the flows run from era to organisation to direction.
At first glance it is just spaghetti. The middle column holds over a hundred organisations, about 115 of them, the ribbons cross, and reading it all at once is impossible. It is also unnecessary. So let us roll it up into big blocks: the roughly 115 organisations into seven actor types (researchers, funders, policy institutes, talent training, industry, meta and community, and government bodies), and the 17 directions into four families:
Foundations and strategy covers macrostrategy, agent foundations, value learning and forecasting.
Technical safety covers interpretability, evaluations, AI control, RLHF, scalable oversight, model organisms, robustness and the like.
Governance and infrastructure covers governance, field-building and disclosure norms.
Capabilities covers the growth of model capabilities itself.
The same picture comes next in two cross-sections that answer different questions: one shows the route as a whole, the other walks it step by step.
Cross-section one, the overall map. Three axes run left to right, from era (the when) to actor type (the who) to direction family (the on-what). The ribbon colour shows at once which family dominated and how that shifted.
The colour shows the direction family, and the numbers count non-money events, so they measure how much was collected rather than the field's real output (caveat 2). A diagonal jumps out. The blue foundations and strategy ribbons crowd the left and the early eras, while the orange technical safety gains mass toward the right. (interactive).
Cross-section two, by era. It is the same flow, with one mini-panel per era. The center of gravity shifts step by step across the panels, and one family dominates each period.
Each panel is one era, running from actor type to direction family, and the ribbon width is the number of events. The blue foundations and strategy rules from 2005 to 2016. Then from the 2020s the orange technical safety dominates the panels. (interactive).
There is also a third cross-section: two clean two-level flows side by side, splitting the when and the who.
On the left the flow runs from era to direction family, showing how the field's focus shifted over time. On the right it runs from actor type to family, showing who works on which family. The ribbon width is again the count of non-money events, which reflects collection effort rather than the field's true volume (caveat 2). On the left, the orange technical safety gains mass toward the later eras. On the right, researchers and industry pull mostly toward technical safety, while government bodies and policy institutes pull toward governance and infrastructure. (interactive).
The era-blocks show where the field's center of gravity moves. The number beside each is how many non-money events landed in that era (collection density, not world volume, see caveat 2):
The prehistory runs from 2005 to 2012. The field is almost empty, just five events: the first macrostrategy institutes (FHI in 2005, GCRI in 2011), MIRI's founding for agent foundations in 2005, and the earliest capabilities and community nodes (DeepMind in 2010, CFAR in 2012). It is a philosophical question with no industry behind it.
Next is the founding phase, 2013 to 2016. Macrostrategy still rules, at 10 of 27 events, but the first technical works and infrastructure start appearing beside it. The key nodes here are FLI, MIRI and the first Open Philanthropy.
Institutionalization follows across 2017 to 2019. There is an explosion of field-building (10 events) and governance (4), and the center of gravity moves to OpenAI, Paul Christiano and GovAI as the field starts building real institutions.
The prosaic turn arrives in 2020 and 2021. Infrastructure still leads, at 6 events, but interpretability and AI control appear for the first time as separate ribbons. The new nodes are Anthropic and Redwood.
Scale and ChatGPT dominate 2022 and 2023. A dense block of 33 events, broken down as field-building (12), governance (5), evaluations (4), interpretability (3) and AI control (2). At the center sit the FTX Future Fund, Anthropic, CAIS and ARC Evals.
Consolidation fills 2024 to 2026. After the exhaustive backfill, this is the densest block in the entire base, with 82 non-money events from 2024 to the first half of 2026. Governance leads with 26 of them, a stretch of summits, laws and commitments. Behind it come the empirical technical families, interpretability (12) and evaluations (12), then robustness (9) and model organisms (6). One node collects more ribbons than any other: Anthropic, with 9 events. And instead of collapsing, the field proliferated. 2024 became a record year for organisations founded, with 24 founding events against 7 in 2025, a wave of narrow specialists. There were interpretability shops such as Goodfire, Transluce, Guide Labs, Tilde, Simplex and Decode. There were AI-security and evaluation shops such as Palisade, Gray Swan and Virtue AI. There was chain-of-thought monitoring at Geodesic. And there was Bengio's "Scientist AI" outfit LawZero. One caveat: the high density of 2024 and 2025 is partly collection density, since the fresh years were backfilled from live primary sources in far more detail than the 2005-2015 retrospectives, rather than purely the growth of the field.
This migration is the storyline of the whole post: from the bottom-left corner (philosophy, lone institutes) to the top-right (empirics, attention and money concentrating around Anthropic and the safety institutes). But concentrated attention doesn't mean concentrated organisations. In this last era the field proliferates into dozens of narrow organisations (2024 was a record founding year in the base). The sections below take that apart era by era.
The field's activity composition shifted too, meaning what the eras are even made of. The prehistory era (2005-2012) is 100% organisation foundings: no papers, no grants yet. Publications, grants and funding statements enter only from 2013 on. By the last era foundings are just about 28% of a much larger, more varied mix.
This shows the shares of the event types, such as foundings, publications, grants and statements, within each era. Caveat 2 still holds: this is the composition of the collection, not the composition of the world. (interactive).
Finally, here's how all of it changed across the eras in one frame: four comparable panels where five epochs run horizontally (the same five substantive Parts below).
The panels run left to right and top to bottom. The first panel shows the direction families as a share of each era's events, out of 100%: the blue foundations and strategy starts at 67% in the 2005-2013 era and shrinks to 6% by 2024-2026, while the orange technical safety climbs to about 60%. Above the columns sits the total number of events in the era, so that normalizing does not hide the field's roughly tenfold growth. The second panel shows the actor types the same way, across seven types. Early on it is almost all researchers, at 56% in the 2005-2013 era. Funders enter as a distinct force in 2014 to 2019, at 24%. And by 2024 to 2026 the mix is transformed: government arrives from nothing to 24% and industry to 26%, now rivalling researchers at 29%. The state and the labs, not just philanthropy, drive the last era's recorded activity. The third panel shows the money by fund, the absolute amounts of all itemized grants per era, roughly $764 million all told. The six long-standing philanthropic funders (OpenPhil, SFF, FTX, FLI, LTFF and Tallinn) keep their own colours, while the newer government and international money (UK AISI at $159 million, ARIA at $74 million, Canada at $36.5 million, NSF at $20 million, Australia at $19.7 million, and so on) rolls into one grey band of other, government and new money. So the growth in money runs consistently end to end: about $0 at the start, then about $128 million in the 2014-2019 era led by philanthropy, then about $242 million in 2022 and 2023 and about $335 million in 2024 to 2026, where the grey government band dominates. The fourth panel shows the organisation dynamics, with births rising and closures and pivots falling. A couple of caveats: the first two panels are shares of collection events (caveat 2), not world volume, and the money panel is itemized grants only, with donor totals and pledges excluded, so that nothing is counted twice against the cumulative-funding chart. (interactive).
The birth and fade of individual directions already show up above on the direction-lifespan chart, and the cross-cutting over-all-years money and attention charts sit together in the Cross-cutting patterns chapter, which comes after the epochs. Now for the same five epochs, in order and in words.
Epoch 1, prehistory and birth (2005-2013): a question without tools
This era holds 9 events in the base, and the birth and fade of its directions runs along the direction-lifespan chart above.
It doesn't start with neural networks. It starts with a philosophical question: how do we avoid wrecking humanity's distant future? In 2005 the Future of Humanity Institute (FHI, Nick Bostrom) opens at Oxford, the earliest reference point in the base. Around it forms what this post calls macrostrategy: the broadest philosophical reasoning about global risks. Over the next years GCRI (2011), CSER (Cambridge, 2013), FRI (2013) and CFI (2015) join in. By event density, macrostrategy peaks in the mid-2010s.
The second root is mathematical. In 2013 MIRI pivots from public outreach to friendly AI as formal mathematics (decision theory, logical induction). That is the birth of agent foundations, the attempt to understand an idealized agent before anyone builds it.
At this stage the table holds almost no money. There's a question and there are first researchers, but no industry and no grant flows. The early nodes (FHI, GCRI, FRI) have already run for years, and none of them has died yet. This shows on the organisation-lifespan diagram in the Cross-cutting patterns chapter, after the epochs.
Epoch 2, institutionalization and the first big funder (2014-2019): money appears
In this era the money panel fills for the first time, and OpenPhil dominates immediately.
The turning point is January 2015: the FLI conference in Puerto Rico ("Future of AI"), which Nate Soares would later call exactly that. It produces an open letter and, with it, the field's first big donor: Open Philanthropy.
On cumulative money by fund (the field-wide summary money charts are in the Cross-cutting patterns chapter) the picture is unambiguous. OpenPhil dominates the whole field's funding. By annual donor totals, its investment into technical safety rises from $1.19 million in 2015 to $6.56 million in 2016, then to $43.2 million in 2017, up to a peak of $81.7 million in 2021. But that 2017 jump is mostly a single grant, the $30 million for general support of OpenAI in March 2017. A spending peak can be one big bet rather than a lot of research.
These same years also give birth to governance: policy, institutes, AI governance. In 2019 OpenPhil puts $55 million over five years into founding CSET (Georgetown), the largest governance grant of that epoch. Value learning runs in parallel, with Stuart Russell's center CHAI (Berkeley, 2016), which OpenPhil immediately backs with $5.6 million over two years.
Where the money actually went, by direction and year, shows up most clearly through itemized grants, without the aggregate donor totals, which would count the same dollars twice. The detailed money-by-track breakdowns live in the Cross-cutting patterns chapter. One figure stands out already: field-building (infrastructure like funds and fellowships) accounts for $326.1 million by itemized grants, the field's largest money flow. That figure grew after backfill, once field-building came to include large government programs like the UK AISI taskforce and new regranters. Next to it the other directions almost disappear.
Epoch 3, the prosaic turn (2020-2021): big models change everything
In this era the families' center of gravity shifts noticeably into technical safety.
By 2020 it's become clear that strong AI, if it arrives, will come from large neural networks rather than from pure agent theory. So the field turns to prosaic alignment, working with real models. Three shifts land at once:
Mechanistic interpretability is born. March 2020 sees "Circuits" (Chris Olah, on distill), then Transformer Circuits in 2021. By the denoised arXiv proxy (mechanistic interpretability, sparse autoencoders and probing, after the bare word interpretability, which had been catching almost all of general ML, was dropped), attention is already rising, from roughly 34 publications in 2020 to 69 in 2021, then climbing steeply to 125, then 257, then 657 by 2025, and that early rise turns out to be only the beginning.
Reward modeling develops into RLHF as a technique, with a clear lineage from Concrete Problems in 2016 to Deep RL from Human Preferences (Christiano and Leike, 2017).
Agent foundations fade. Abstract, deep-learning-agnostic agent theory stops convincing donors in an era where empirical work on LLMs decides everything. Discrete events on the track thin out after 2021 and leave only a lone 2024 marker (MIRI's pivot). The direction survives mostly as background, not as fresh events.
The shifting money leader year to year, and the drift of shares from early technical bets toward governance and infrastructure, both show up on the cross-cutting money chart (absolute money by track). It sits in the Cross-cutting patterns chapter, so it reads across the whole field rather than one era at a time. The publication attention curves (arXiv proxy) by track sit there too, and the interpretability curve on them starts right around 2020-2021.
Epoch 4, the ChatGPT moment and the FTX shock (2022-2023): money, institutes, a break
In this era the money now comes from several funds at once (SFF, OpenPhil, FTX), and among the actors, institutes and industry noticeably grow.
In late 2022 ChatGPT pushed AI into big politics and mass consciousness. In the table that surfaces as a surge on several fronts at once:
Evaluations, the dangerous-capabilities evaluations: ARC Evals (Beth Barnes, 2022), then Apollo Research (2023), pre-deployment evaluations of GPT-4 and Claude, and the UK and US safety institutes in 2023, around the Bletchley summit.
AI control, the Redwood direction. The organisation was founded in June 2021, and by 2023 it has a formalized control research agenda. The money is dense: OpenPhil gave Redwood $9.4 million in 2021, $10.7 million in 2022 and $5.3 million in 2023, plus another $6.6 million from FTX in July 2022.
The governance explosion. On the governance subtrack the money rose from $0.4 million in 2018 to $1.5 million in 2020, then $3.6 million in 2021, $10.3 million in 2022 and $18.4 million in 2023, roughly a 46-fold rise over five years. For comparison, technical safety in 2023 is $24.6 million, still larger.
Then the shock. November 2022, FTX collapses. In half a year (February-August 2022) the FTX Future Fund had handed out $18.7 million by name to AI safety, about $32 million by estimate overall. Now it's cut off, and some grants even have to be clawed back. On the cross-cutting cumulative chart the FTX line stops at 2022 and never grows again. The births and closures of organisations by year sit on the summary organisation-dynamics chart there too, and in the organisation-dynamics panel of the by-era overview chart.
Epoch 5, consolidation and the ending (2024-2026): Safety disappears from the names
In this era, consolidation looks more like continued activity, with the organisation dynamics showing a record 31 births against 8 closures or pivots.
The last era in the data is contraction and redefinition.
In April 2024 FHI closes, the institute where it all began.
In May 2024 OpenAI disbands the Superalignment team, which had been announced in July 2023 with a promise of 20% of compute over four years, after Sutskever and Leike leave. The direction scalable oversight loses its main institutional backer.
In 2024 MIRI, the ancestor organisation of agent foundations, officially pivots from technical research to communications and policy. The oldest fundamental direction leaves the research stage, though not because it was solved: its carrier organisation simply changed its mission.
In 2025 the word Safety disappears from the safety institutes' names. In February 2025 the UK AI Safety Institute becomes the AI Security Institute, and the US AI Safety Institute at NIST becomes CAISI, the Center for AI Standards and Innovation. Safety gives way to standards and to safety as security.
In November 2025 the main funder changes its name. Open Philanthropy renames itself Coefficient Giving, announced on 18 November 2025, and restructures into 13 thematic multi-donor funds instead of one anchor donor in Good Ventures. The very money source that started the field's institutionalization changes its own name, echoing the disappearance of Safety from the institute names.
But this consolidation is really specialization and proliferation. 2024 is the record year in the base for organisations founded (24 founded-events). Interpretability gets commercialized and splinters into startups (Goodfire, Transluce, Guide Labs, Tilde, Simplex, Decode Research). A separate AI-security and evaluations cluster emerges, in Palisade Research, Gray Swan AI and Virtue AI. Organisations for fresh sub-agendas keep appearing: Geodesic for chain-of-thought monitoring, Luthien for practical AI control, Softmax for multi-agent alignment, Formation Research for lock-in risk, and Yoshua Bengio's LawZero in June 2025, built around a non-agentic "Scientist AI" as an oversight layer. Even the giants open internal cells: in 2025 Meta sets up a "superintelligence alignment and safety" team. Alongside all of it, non-technical infrastructure proliferates too: the forecasting cell AI Futures Project (Kokotajlo, the "AI 2027" scenario), the watchdog AI Lab Watch, the standardizers AI Standards Labs, national centers (Beijing Institute of AI Safety and Governance, France's CeSIA), the international association IASEAI, and the AI Whistleblower Initiative.
Governance stops being background and becomes the era's main storyline. It accounts for a third of the block's events (26 of 82 non-money), and the chronology tightens from summit to summit. November 2023, Bletchley. Then the AI Seoul Summit (May 2024): 16 frontier companies sign the Frontier AI Safety Commitments, each promising to publish its own safety framework with risk thresholds, and 27 countries plus the EU, in the Seoul Ministerial, agree for the first time to develop common thresholds for severe risks. 1 August 2024, the EU AI Act comes into force, the world's first broad, horizontal AI law, with rules for GPAI from August 2025 and a GPAI Code of Practice. In the US, hard law stalls. California's SB 1047 (tests, a kill switch, liability) is vetoed by Newsom (September 2024). Then in January 2025 the federal course swings from safety to dominance: the repeal of Biden's AI executive order, the "Removing Barriers to American Leadership in AI" order. Later in 2025 California comes back with a softer SB 53, focused on transparency. The industry builds its own: the Frontier Model Forum in 2023 and its AI Safety Fund, worth over $10 million in rounds of $4 million in November 2024 and over $5.2 million in December 2025. And an IPCC for AI takes shape as the International AI Safety Report, which ran from an interim version in May 2024 to a full report in January 2025 under Bengio, landing just before the AI Action Summit in Paris in February 2025, where the tone has already slid from safety toward action and deployment.
The technical directions don't stand still. Interpretability gains new momentum: Scaling Monosemanticity (Anthropic, May 2024, "Golden Gate Claude"), Gemma Scope (DeepMind, July 2024), an open set of SAEs shipped as public infrastructure, circuit tracing and "On the Biology of a Large Language Model" (March 2025), and "Auditing Language Models for Hidden Objectives" (Anthropic, March 2025), the first real audit game, where a goal is hidden in a model and then uncovered blind. The youngest direction, model organisms, forms around the empirics of deception: Sleeper Agents (January 2024), Sabotage Evaluations (Anthropic, October 2024), Apollo's "In-Context Scheming" (December 2024, where scheming showed up in 5 of 6 frontier models), Alignment Faking (December 2024), and Agentic Misalignment (Anthropic, June 2025, where 16 models in stress-tests resorted to blackmail or leaks to avoid shutdown). Safe-scaling frameworks evolved into their own genre. DeepMind published the Frontier Safety Framework in May 2024, and Anthropic revised its Responsible Scaling Policy that October. Scalable oversight had lost its home when the Superalignment program was cut, so researchers rebuilt its goals in other projects. In December 2024 OpenAI introduced Deliberative Alignment. The following September, it teamed up with Apollo to release anti-scheming training that reduced hidden actions by about 30-fold but never eliminated them. That work spurred agentic control evaluations, among them Redwood's Ctrl-Z in April 2025. Everything culminated in July 2025, when OpenAI, DeepMind, Anthropic and the UK AISI co-signed the position paper "Chain of Thought Monitorability", which outlined their collective responsibility to protect the limited window into model reasoning. The first six months of 2026 followed similar patterns, and the dataset includes only data through H1.
Interpretability continued to advance. On 7 May 2026 Anthropic published "Natural Language Autoencoders", which turns a model's internal activations into readable text. In the pre-deployment audit of Claude Opus 4.6, the method identified evaluation awareness: the model recognized it was under test in 16 to 26% of benchmark transcripts, but under 1% of real-world traffic. The discipline is now part of the release cycle rather than a side project.
Evaluations turned toward security. The UK AI Security Institute released a report stating that Claude Mythos Preview was the first model to complete the 32-step network-attack scenario "The Last Ones" on 13 April; GPT-5.5 followed on 30 April. The 2025 rebranding from Safety to Security marked a genuine change, because the agenda shifted to autonomous cyber risk. In May 2026 the EU provisionally approved the "Digital Omnibus" as its first set of AI Act amendments, without changing the obligations for GPAI models.
On the political front, in May 2026 the EU preliminarily agrees on the "Digital Omnibus", the first amendments to the AI Act, though the obligations for GPAI models are left untouched.
Cross-cutting patterns: money, attention and the three fates of a track
Having walked the field epoch by epoch, it is worth stepping back to look across all the years at once, because some patterns surface only that way and slicing them by era hides them. Money is the most tangled of these, so it comes first.
Four money views, and why the same fund shows different amounts. The money here gets counted four different ways, which deliberately do not add up; otherwise the same dollars would land in the total twice. They are four views of one pile.
The first is the itemized grants, at $763.6 million. These are the real targeted grants, recorded as they went out. They show in full on the money-by-track chart, where the bars are the tracks plus a grey other-and-untracked strip of $83.6 million, and on both sankeys. Only the track-classified $680.0 million appears on the radar and the per-track money-versus-attention chart, because the aggregate donor baskets and one capabilities grant, $83.6 million together, have no specific track. The second view is the donor annual totals. Each is a donor's full budget for the year, the umbrella figure rather than the sum of its sub-category breakdowns, which would tally the same money twice. This is howOpenPhil (about $304.5 million) and LTFF (about $3.64 million) show up on the cumulative-money-by-fund chart, while every other fund shows by its itemized grants. The same two totals run to 2024 on the field-wide money-versus-attention chart; the 2025-and-later donor totals are not published yet, but the individual-grant view keeps flowing past that point.
The third view is the venture money, the VC and equity, at $268.5 million. It expects a return, arrives across five rounds, appears only on the third panel of the cumulative-money chart, and is never summed with the first two. The fourth is the pledges, budgets and estimates, kept as context and never summed. This is money that is not a disbursed grant: FTX's launch pledge of about $160 million against the $18.7 million it actually disbursed, organisations' annual budgets like MIRI, CHAI and CSER, and overlapping field-wide estimates such as the roughly $40 million EA spent in 2019. It was invisible on every chart until it got its own dots-only view. Because the field-wide estimates already include the organisation budgets, it is heterogeneous and non-additive, so it is plotted as points and never totalled, purely so that nothing collected stays hidden.
Here is where the confusion comes from. OpenPhil is $176.77 million as the sum of its itemized grants under the first view, and about $304.5 million as the donor annual total under the second. It is the same organisation seen two ways. No figure is lost; the two simply cannot be added together.
Money across the whole field. The historical main funder was OpenPhil, with $176.77 million of itemized grants, and after backfill it got caught and overtaken by the government money of half a dozen countries. The UK AISI and its taskforce come to about $159 million altogether (£100 million plus the Alignment Project), with ARIA Safeguarded AI at $74 million close behind. Then a longer tail: Canada CAISI at $36.5 million, NSF Safe Learning-Enabled Systems at $20 million, Australia AISI at $19.7 million, the US AISI at NIST at $10 million, DARPA GARD at $10 million, and the EU AI Office at $9.8 million. SFF stands alongside at $144 million. FTX still cuts off at the end of 2022. So the one-big-funder picture of 2020 to 2022 gives way to something far more distributed in 2023 to 2025, and noticeably more governmental, now international. But this is already different types of money (caveat 5). And beside it a fourth type has appeared, venture capital into safety startups, about $268 million of equity. It stays a separate view with its own colour and line across the charts below, never mixed into the grant totals. The grant money appears three ways, ranked, cumulative over time, and as a dot-strip, because no single view catches both who gave how much and when the money arrived.
There is one bar per funder, sorted, on a log axis, and coloured by type. The philanthropy bars are blue: Open Philanthropy sits at its donor annual total of*$304.5 million** (the same OpenPhil that reads $176.77 million under the itemized-grant view; caveat 5), then SFF at $144 million, FLI at $39 million and FTX at $18.7 million disbursed, followed by Schmidt, Longview, Manifund, Tallinn, LTFF, Founders Pledge and Effektiv Spenden. The government bars are orange: the UK AISI at about $159 million, ARIA at $74 million, Canada CAISI at $36.5 million, NSF at $20 million, Australia AISI at $19.7 million, the US AISI at NIST at $10 million, DARPA at $10 million and the EU AI Office at $9.8 million. The corporate bar is red: the AI Safety Fund at the Frontier Model Forum, at $9.2 million. And a distinct fourth group, the VC and equity money, is green: Protect AI at $108.5 million, HiddenLayer at $50 million, Goodfire at $50 million, Gray Swan at $40 million and Lakera at $20 million, about $268 million in all. Equity expects a return, so it is never summed with the $763.6 million of grants. It just shares the magnitude axis, which shows safety becoming a market. (interactive).*
The same money over time, now stacked by funder type, showing not only who but when.
The stack is cumulative grant dollars by type, with philanthropy in blue, government in orange and corporate in red. OpenPhil and LTFF sit at their donor annual totals and the rest are itemized, which is why the canonical stack runs a little above the $763.6 million itemized-grant total. The shape is the point here: through 2020 it is almost entirely a single blue channel of philanthropy. Then from 2023 the orange government band explodes, with half a dozen national safety institutes landing at once, turning one fragile channel into a broad, increasingly governmental and international flow. The dotted green line is the VC and equity money, about $268 million, a separate view laid over the same axis and never summed into the grant stack. (interactive).
Third, a dot-strip, since the shapes above hide the rhythm of individual moves.
The x-axis is the year and the y-axis is the funder, sorted by total, while the dot area shows that year's dollars and the colour shows the type. OpenPhil is the only funder active almost every year, and its 2021-2023 dots are the largest. The government cluster is unmistakable in 2023 to 2025, in big orange dots for the UK AISI, ARIA, Canada CAISI, NSF and Australia AISI. And VC rounds show up as green diamonds from 2023 onward, with Protect AI in 2024 the largest. The timing makes the regime change legible: philanthropy carried the field's money alone until 2023, and then the state and the market arrived together across the last three years. (interactive).
A fourth view keeps any collected dollar from staying invisible: the money that is not a disbursed grant.
Ranked horizontal bars group the sub-types on a log axis. What matters here is the magnitude, because these are not comparable, additive dollars. The colour shows the sub-type. The launch pledges include the FTX Future Fund's pledge of about $160 million at launch. The organisations' annual budgets include FTX's own $50 million and $32 million figures and MIRI at about $7.5 million a year, along with CHAI, CSER, Ought and Lightcone. The field-wide estimates include the estimate of roughly $40 million spent on AI safety in 2019 plus the 2014-2016 estimates, which deliberately overlap the organisation budgets. And the seed at founding includes SFF at about $2 million and Timaeus at $0.14 million. This view is heterogeneous and non-additive. The field estimates already contain the organisation budgets, and a pledge is not a disbursement. So it is never summed and never mixed with the grants ($763.6 million), the donor totals, or the VC money ($268.5 million). Its only job is to show that these numbers exist in the data. (interactive).
That gap shows up most sharply in one case, which gets its own chart.
Here is the headline as a dumbbell: the FTX Future Fund pledged about $160 million at launch but actually disbursed only $18.7 million before its collapse in November 2022. That gap between what was announced and what was actually delivered is the clearest warning of the era. Below it, the rest of the non-grant money, the organisation budgets, the overlapping field estimates and the seeds, is ranked on the same log axis for scale. It is still the fourth view, so never summed with grants, donor totals, or VC. (interactive).
And now for where this money actually flows, the routing of itemized grants from fund to track.
The flow here is a Sankey of the known itemized grants, with funds on the left, research tracks on the right, and the ribbon width showing the grant amount. OpenPhil and SFF clearly give across many tracks at once, while the government institutes (UK AISI, Canada CAISI, Australia AISI, US AISI at NIST, EU AI Office, DARPA and NSF) enter with precision, mostly into field-building, evaluations and robustness. This routes all the itemized grants, some $764 million altogether: grants whose direction is an aggregate donor-basket (the mixed-technical or technical-safety baskets) or capabilities get their own right-hand nodes rather than being dropped. Only donor totals and the VC and equity money stay out, because otherwise those dollars would be counted twice. (interactive).
The same grant flow, but now with a third level, the era, added on the left. That way it shows who, where and when at the same time.
This has three levels, running from era to fund to track. It reads clearly how the mass of grants shifted into the consolidation era of 2024 to 2026, and exactly how the new government institutes support evaluations, field-building and robustness. The recipient organisation is free text in the detail field, not a structural column, so it does not get its own level, a gap in the data. (interactive).
Money versus attention, and the life of organisations. Money, as donor totals, and scientific attention, as the arXiv proxy, keep drifting apart over time. And the births and closures of organisations give a separate organisation-dynamics curve across the whole field.
Four things are drawn here, and it is worth reading them separately.
The blue bars, on the left axis in dollars per year, are the annual donor totals of the two funders that publish a clean field-wide number, Open Philanthropy and LTFF, de-duplicated by taking the umbrella mixed-technical row where it exists, otherwise that funder-year's full total, so the sub-category rows never get added on top. This is the only yearly money series that never counts a dollar twice, which is exactly why it is used here.
As for why the bars stop at 2024 and there is nothing for 2025 or 2026: this series depends on the donors' published annual giving totals, and Open Philanthropy and LTFF simply have not released their 2025 and 2026 totals yet, since the annual report lags. The last bar, in 2024, is OpenPhil's technical-safety total of about $28 million. So the empty 2025 and 2026 mean missing source data, not a build or cache glitch, and not that the money dried up. The itemized-grant charts above, the era shift and the money-by-track, do run into 2025, because they count individual itemized grants, a different view that must not be summed with these donor totals, since that would count the same dollars twice.
The solid red line, on the logarithmic right axis, is the deduplicated safety corpus, one combined arXiv OR-query with each unique safety paper counted once per year. This is the reliable field-level line.
The dotted red line is the inflated keyword-proxy sum over the non-overlapping core of tracks, drawn on purpose to show how much the per-track proxies overstate the real count by counting papers twice when they match several tracks (caveat 1).
The grey band marks the partial final year, the first half only, and the right axis is logarithmic. Both money and attention are proxies, but the deduplicated line is reliable at the field level. (interactive).
Up is new organisations, down is closures and focus changes. One thing to keep in mind: this counts events per year, not organisations. There are 14 closure or pivot events in all, 3 closures and 11 pivots, and a single organisation can change focus several times. MIRI pivoted in 2013, 2018 and 2024; OpenAI pivoted in 2019 and then closed its Superalignment team in 2024; and Open Philanthropy pivoted in 2024 and 2025. So there are more red events in total than distinct dead organisations, which is why there is more red here than there are crosses on the chart below. (interactive).
Each bar is one organisation, unlike the chart above, which counts events: it runs from the founding year to the last closure or focus change, or to the present if still alive, with a cross marking the organisation's last such event. The founding dates of MIRI (2005, when SIAI reoriented to AI risk), Open Philanthropy (2014), CFAR (2012), LTFF (2017), METR (2022, as ARC Evals) and the FTX Future Fund (2022) are added from sources. So now every organisation with a closure or pivot is visible, and the chart converges with the one above. The 14 red events there collapse to 10 distinct organisations here, each carrying one cross on its last such event, because MIRI, OpenAI and Open Philanthropy each pivoted more than once. Pivots happen across the whole timeline, with MIRI's turn coming as early as 2013 and CFAR's in 2016. The actual closures, just 3 of the 14 events, land in 2024, at FHI and OpenAI's Superalignment team, with the FTX Future Fund's 2022 collapse as the earlier exception. (interactive).
A track's three fates. The table's biggest lesson is that a direction here can meet three different fates, and disappearing from the conversation is not the same as dying.
Rising: interpretability, evaluations and AI control, all born recently and growing. Interpretability is the reference case. By the denoised arXiv proxy, after the bare word interpretability was stripped from the query, where it had caught all of general ML, the curve is far more modest, rising from 69 in 2021 to 81, then 125, then 257, then 657 by 2025. The shape holds, a steady exponential rise, and it is that shape, not the absolute count, that carries the point. The same climb shows in the field-wide deduplicated corpus, which runs from 351 in 2021 to 427, then 890, then 2191, then 3813 by 2025 unique safety papers per year, almost an order of magnitude in four years. Money, meanwhile, lags sharply behind attention, with only $1.04 million of itemized grants into interpretability, the classic case of attention outpacing its funding. For evaluations the gap closed almost overnight. Before 2024 there was no itemized money on the track at all. Then it filled to about $96.7 million in itemized grants, nearly all of it in 2024 and 2025, making evaluations one of the field's better-funded tracks in a single year. Some of that is dedicated philanthropic and US money: the AI Safety Fund at the Frontier Model Forum gave $9.2 million in two rounds, $4.0 million in 2024 plus $5.2 million in 2025, for biological, cyber and agentic evaluations; the US AI Safety Institute at NIST gave $10 million in 2024; and Schmidt Sciences gave $10 million in 2025 for the science of evaluations. But the bulk comes from the new international government institutes routing into evaluations: Canada CAISI at $36.5 million, Australia AISI at $19.7 million and the EU AI Office at $9.8 million, plus an OpenPhil grant of $1.5 million. In the per-track panels below, evaluations now shows up with both a money bar and an arXiv line, and on the gap chart it sits close to the field average instead of off the money-starved end.
Absorbed: reward modeling, now RLHF, and value learning. RLHF by the arXiv proxy, on the broadened query for RLHF, reward models, DPO and preference optimization, runs from 25 in 2021 to 43, then 275, then 1010 in 2024, then 1777 in 2025. The direction vanished from safety conversations because it had won. RLHF and DPO became the standard way to fine-tune every commercial LLM and dissolved into mainstream ML. Publications keep growing precisely because it is shared infrastructure now, not a separate safety track.
Faded: agent foundations and early macrostrategy. They were the field's core, then shrank. Agent foundations barely registers on the arXiv proxy before 2024, since the term is young while the direction itself is old, the reverse case of proxy lag. The closure of FHI in 2024 is a symbolic full stop for macrostrategy.
Money and attention for each track, laid out one small panel at a time so nothing overlaps:
The grid holds small multiples, one mini-panel per track, ordered by total grant money. In each panel the bars, on the left axis, are that track's itemized grant money per year in dollars, and the dotted line, on the logarithmic right axis, is its scientific attention on the arXiv proxy for the same years. So each track is legible on its own and its divergence between money and attention reads directly. This replaces the old single dual-axis chart, where about 20 attention lines piled over the stacked bars into unreadable spaghetti. Here all the itemized grants show, about $764 million in total: grants without a specific track collect into one grey other, untracked or aggregate panel, about $83.6 million, made up of the aggregated donor baskets for mixed-technical and technical-safety plus one non-safety grant, money only, with no arXiv line, so no dollar is lost and the grid still totals the full $763.6 million. Donor annual totals for OpenPhil and LTFF, and the venture money in equity, live on the cumulative-funding chart and are not summed here, because that would count them twice. Interpretability shows a steep dotted rise over tiny bars, its attention far ahead of its money, even after denoising the curve by dropping the bare word interpretability. Evaluations shows a wall of bars for 2024 and 2025. AI control has money bars from 2021 before its attention lifts. Tracks that have an arXiv proxy but no itemized grant money at all (reward modeling, red-teaming, unlearning, chain-of-thought faithfulness, honesty and ELK, and the umbrella and nested directions, per caveat 1) get no panel here, since there is no grant bar to rank them by. Their attention instead sits on the field-level money-versus-attention chart and the per-track proxy curves. Both series here are only proxies, and the deduplicated safety corpus is deliberately not drawn on this grid, being field-level. (interactive).
This diverging gap chart replaces the old log-log trajectory tangle. For every track with both a money series and an arXiv proxy, the bar shows how far it sits from the field-average rate of money to attention, computed as the base-2 logarithm of the actual attention divided by the field's average papers-per-dollar times that track's money. A blue bar pointing right means scientific attention runs ahead of money.A red bar pointing left means money runs ahead of attention. The label gives the factor. Interpretability is the extreme blue case, its attention roughly 36.9 times ahead of what its modest $1 million or so in grants would predict, the classic science-runs-ahead-of-money gap. On the red side sit scalable oversight, agent foundations and governance, directions where itemized money outran the arXiv attention. Both signals are proxies, per caveats 1 and 2. (interactive).
A companion dumbbell view lays out the same tracks. Instead of the magnitude of the gap it shows the ordering: each track's rank by grant money, as an orange dot, against its rank by arXiv attention, as a green dot, joined by a connector, where rank 1 is the smallest and the largest rank is the biggest. A long blue connector means attention ranks the track far above where money does. A long red one means money ranks it far above attention. Overlapping dots mean the two agree. Set beside the gap chart above, which shows magnitude, it makes the who-leads story concrete: interpretability's dots pull wide apart, while better-matched tracks keep their two dots close, both ranks resting on proxies (caveats 1 and 2). (interactive).
And here is the money footprint of the tracks, on a single money axis, with a caveat.
A note on this one: it is a single logarithmic axis for money, not a multi-metric comparison. Multi-metric radars mislead across different scales, so there is no one built here. (interactive).
Finale: open questions
There is no moral to end on; the data does not hand one over. What it does leave is a handful of open questions the field is currently standing on:
If Safety got dropped from the institutes' names, is that a change of sign or a change of substance? In the last era governance became the dominant flow (26 of 82 non-money events across 2024-H1 2026, a third of the block): summits, laws, commitments. But it all moved under the banner of security, standards and action. Will the same researchers grow up under that banner as under safety, or different ones?
Is consolidation concentration, or is it actually fragmentation? By money and attention, the field is clearly gravitating toward a few names: Anthropic, the safety institutes, one large donor. When measured by headcount, the situation reverses. The year 2024 set a record for new organizations, mostly shops with one to five employees in interpretability, AI security and chain-of-thought monitoring. Is the field maturing and specializing, or fragmenting, with only a few likely to survive? Does absorption count as success, or does it erase the original players? RLHF won, then was reabsorbed into ordinary engineering. Is it good news when a safety method stops counting as a safety topic? Once that happens, who keeps track of where the method fails?
How much does the field depend on one or two funders? When shown together, the OpenPhil channel and the FTX collapse reveal the field's exposure. Evaluations only recently gained an independent backer, the AI Safety Fund, yet interpretability still leans almost entirely on a single funder. What if that funder drops out or scales back support? Scientific progress has outpaced funding for interpretability; governance funding, by contrast, has grown faster than the science. Which of those two gaps is the healthy one, and what does the answer reveal about what the field is actually doing?
Science runs ahead of money in interpretability, while governance money ran ahead of publications. Which gap is the right one, and what does it tell us about what the field is actually doing?
Appendix: methodology
Data and sources. The dataset includes 323 documented events from 2005 through mid-2026, plus a deduplicated safety corpus and arXiv proxies across 23 directions. Each count links to the exact search-query URL that reproduces it. We backfilled most of the recent detail through several passes over live primary sources, rather than retrospectives that stop at 2023. We identified newly founded organizations from an analysis of field growth on the EA Forum. The research milestones for 2024 and 2025 come from the "Shallow Review of Technical AI Safety", with its arXiv and lab links, and the governance events from primary sources: gov.uk, EU documents, California legislation, and White House records.
They brought in new philanthropic grantmakers (Manifund, Longview, Founders Pledge, Schmidt Sciences, Effektiv Spenden, Macroscopic Ventures and Nonlinear) and international government institutes (UK AISI and ARIA, US AISI at NIST, NSF, Canada CAISI, Australia AISI, EU AI Office, DARPA's GARD and AIxCC, IARPA's TrojAI, Singapore AISI, the annual UK AISI budget of about £50 million a year, Germany's DE-AISI and India AISI). They also added venture rounds into safety startups as a separate view, recorded as investments: Goodfire at $50 million, Gray Swan at $40 million, Lakera at $20 million, HiddenLayer at $50 million, and Protect AI at $108.5 million before its Palo Alto acquisition of $634.5 million. And they added the missing founding events for safety organisations (EleutherAI, Apart Research, Lakera, Protect AI) and the individual donor Craig Newmark. The last two enter as a statement and an investment so that grants do not double-count.
In the final pass the arXiv method turned toward precision over breadth. A deduplicated safety corpus was added, one combined OR-query with each paper counted once. The two worst tracks were denoised: the bare word interpretability was dropped from the interpretability track, and truthfulness was flagged high-noise and pulled from the field sum. Every proxy track now carries a noise flag.
Three collector scripts write the data with a source URL and rerun reproducibly: one pulls arXiv, one the LTFF annual payouts from EA Funds, and one the SFF annual totals. So 2024 comes out as the densest year of the base, partly a matter of collection density, which the direction-lifespan charts flag. Every event ties to a primary source, and the source types run as follows.
arXiv preprints give the scientific attention by direction.
Lab technical reports and blogs give Circuits on distill, Transformer Circuits and other Anthropic publications, plus OpenAI and DeepMind announcements.
Community retrospectives and reviews come from LessWrong, the Alignment Forum and the EA Forum.
Nonprofit tax filings, the 990 forms via ProPublica, give the revenue and size of nonprofits.
The web archive, the Wayback Machine, covers pages that have since disappeared or been rewritten.
Regulator pages cover the UK AI Safety and Security Institute, NIST and CAISI, and the EU institutes and documents.
Fund announcements and annual reports come from Open Philanthropy, the Survival and Flourishing Fund (SFF), the Long-Term Future Fund (LTFF), the Future of Life Institute (FLI) and the FTX Future Fund, plus a community donations aggregator.
How the money is counted. Itemized grants (a specific amount to a specific recipient in a specific year) are kept separate from donor annual totals (how much a fund allocated in total for a year). To avoid double-counting, each donor uses one canonical series: annual totals for OpenPhil and LTFF, itemized grants for everyone else. The key amounts from the text:
The itemized grants come to $763,625,573 in total, across 79 grants, and by fund they fall into three kinds of money. The philanthropies are the oldest and largest block: OpenPhil at $176.77 million, SFF at $144.04 million, FLI at $39.01 million and FTX at $18.74 million, then Schmidt Sciences at $10.0 million, Longview at $9.0 million, LTFF at $6.41 million, Manifund at $5.42 million, Jaan Tallinn at $5.0 million, Founders Pledge at $0.81 million and Effektiv Spenden at $0.18 million. The government institutes are the newer arrivals: the UK AI Safety Institute at $159.0 million, ARIA at $74.0 million, Canada CAISI at $36.5 million, NSF at $20.0 million,Australia AISI at $19.73 million, the US AI Safety Institute at NIST at $10.0 million, DARPA's GARD at $10.0 million and the EU AI Office at $9.81 million. And one corporate source stands beside them, the AI Safety Fund at the Frontier Model Forum at $9.20 million. The bolded items came in with the last deep backfill; they are already international government institutes (Canada CAISI at CAD$50 million over five years, Australia AISI at A$29.9 million over four years, the EU AI Office at €9.08 million, and DARPA's GARD at $10 million for fiscal year 2024), alongside the new retail fund Effektiv Spenden. OpenPhil also grew thanks to itemized 2024 grants to CAIS, Redwood and MIRI of about $3.8 million, flagged as a subset of the donor total so as not to double-count. These are different types of money (caveat 5), and some of it is stated program budgets (the UK's £100 million, ARIA's £59 million, Canada's CAD$50 million and Australia's A$29.9 million) rather than disbursed amounts. Currencies convert to dollars at reference rates (about 1.25 for the pound, 1.08 for the euro, 0.73 for the Canadian dollar and 0.66 for the Australian dollar) and are flagged in the detail field. Double-counting is excluded: donor totals for OpenPhil and LTFF and their itemized grants are never summed, and the overlaps (NSF against OpenPhil at $5 million, the Alignment Project against OpenAI and Schmidt, and OpenPhil's itemized grants against its total) are flagged in the data.
The VC and equity money is a separate view and is not in this $763 million. Venture rounds into safety startups, recorded as investments, are not charitable grants but investments expecting a return, so they stay separate and never sum with grants: Goodfire at $50 million in a Series A for interpretability, Gray Swan at $40 million, Lakera at $20 million, HiddenLayer at $50 million, and Protect AI at $108.5 million before it was acquired by Palo Alto Networks for $634.5 million, coming to about $268 million of disclosed equity. Tellingly, the very first venture check into pure interpretability, Goodfire's $50 million, rivals OpenPhil's entire annual technical budget. Safety is becoming a market, not only a philanthropic cause.
SFF by year, from verified SFF announcements as recorded in the event base and plotted on the charts, rises from $5.45 million in 2020 to $19.4 million in 2021, then $18.1 million in 2022, then a peak of $42.3 million in 2023, then $24.0 million in 2024 and $34.9 million in 2025, for $144.04 million in total. The SFF annual totals and the LTFF annual payouts export to two separate files, each with a source URL.
OpenPhil's 2024 spend on technical safety is about $28 million as recorded in the grants database, with a caveat: this uses a different counting method than the donations aggregator did for 2015 to 2023, and the organisation itself says "about $50 million" for 2024. For 2023 it is about $46 million on AI safety. In 2025 more than $1 billion moved across all causes, though the fund does not publish an AI-specific total, and of that a $40 million request for proposals on technical AI safety was announced.
On 2026 (incomplete year, H1). Real events of the first half of 2026 came from primary sources (Anthropic Natural Language Autoencoders, the UK AI Security Institute cyber evaluations, the EU Digital Omnibus). But there's no distributed money for 2026 yet. SFF-2026 is only announced at $20 to $40 million with distribution in the autumn, so the amount isn't recorded, and the AI Safety Fund round with a figure fell in December 2025.
The arXiv series were rebuilt with one unified method across all 23 directions: the same reproducible recipe, OR-synonyms in the title and abstract plus a per-track category filter, counting by submission date, in a common 2015-2026 window, dropping the earlier mixed methodology. The per-track proxies gained the deduplicated safety corpus, one combined export with each paper counted once, which on the money-versus-attention chart is the solid line against the dotted inflated keyword sum. The two noisiest curves were denoised: interpretability without the bare word interpretability, where the count dropped severalfold, and truthfulness flagged as high-noise and removed from the field sum.
2026 stays incomplete, the first half only, and is marked with a grey zone. So an incomplete year does not read as a decline, and the last-full-year arXiv diamonds on the direction-lifespan chart sit on 2025. The rename from OpenPhil to Coefficient Giving is pinned to its real date, November 2025.
Five caveats that the above depends on:
arXiv publication counts are a proxy, not bibliometrics. A keyword like RLHF or scalable oversight surfaces on arXiv later than the real start of a track. Broad phrases such as AI control, dangerous capabilities, value learning and model editing add noise on top. And the query itself kept widening with each backfill. An exact phrase gave way to synonyms joined by OR under a category filter; the search then ran across both the title and the abstract, with the category set tuned track by track. So the adversarial track drew on cs.CR and cs.CV, governance on cs.CY, and multi-agent on cs.RO, all inside one 2015-2026 window. That inflates the per-track numbers. Much of the apparent jump is the net widening, not the field growing. Hence the shift in the last pass to precision over breadth: a deduplicated safety corpus, one combined OR-query of safety-specific phrases where arXiv counts each paper exactly once. That is the number to trust at the field level. The two worst tracks were denoised too. Stripping the bare word interpretability from the interpretability query, where it had swept in almost all of general ML, cut the count severalfold. The truthfulness track was worse still: in practice just hallucination and factuality, and so plain NLP, it was flagged high-noise and dropped from the field sum. Every proxy track now carries an explicit noise flag of low, medium or high. The scale is worth stating plainly. The cs.AI, cs.LG, cs.CL and stat.ML papers from 2015 to 2026 run into the hundreds of thousands, while those that self-label as AI safety or alignment number only a few thousand. So even the whole safety corpus is a thin strip of general ML, and the per-track sums overstate it many times over. One last wrinkle. Some proxy tracks are umbrellas, or nested inside one another: the broad alignment track covers everything, constitutional AI sits inside RLHF, and activation steering and singular learning theory sit inside interpretability. The attention curves therefore cannot be naively summed. A field-attention sum uses either the deduplicated corpus or the non-overlapping core of tracks, with the rest shown only as separate curves. What matters is the shape of each curve, whether it bursts or plateaus, and the relative comparisons, not the exact number.
The number of events per track is a biased measure. It reflects how much was recorded, not how much actually happened in the world. For real activity the better guide is money and publications, not the event counter. (On the collection-density chart the recent years are explicitly hatched as under-collected.)
The causal why is interpretation. It is layered on top of verified events, not lifted from a source, and stands as an argument open to dispute.
The grant-recipient organisation is not broken out as a separate structure. It exists only in the event description text, so the money flows run from fund to direction, not from fund to organisation.
The money now mixes several types, and each stays in its own view. After several backfills the money charts stopped being only philanthropic grants. They now also take in government programs from half a dozen countries (UK AISI, ARIA, US AISI at NIST, NSF, Canada CAISI, Australia AISI, EU AI Office, DARPA GARD) and corporate-philanthropic sources (AI Safety Fund, Schmidt Sciences, Manifund, Longview, Founders Pledge, Effektiv Spenden). These are not the same kind of dollar: a government budget is not a private grant, neither one is a compute promise, and some entries are a stated program budget rather than a disbursed amount. Next comes a genuinely new, separate view, the venture money (VC and equity) flowing into safety startups, recorded as investments. Goodfire raised $50 million in a Series A for interpretability; Gray Swan raised $40 million, Lakera $20 million and HiddenLayer $50 million; and Protect AI raised $108.5 million before Palo Alto acquired it for $634.5 million. That is about $268 million of disclosed equity, and it is not a charitable grant: the investor wants a return, so it is never summed with grants and stays out of the grant dollar-charts entirely, in a view of its own. A fourth kind of money is neither grant nor equity: pledges, annual budgets and field-wide estimates. This is money that gets collected without being a disbursed grant, like FTX's launch pledge of about $160 million against the $18.7 million it actually paid out, or organisations' annual budgets, or estimates such as the roughly $40 million spent on AI safety in 2019. It used to be invisible on every chart; now it has its own dots-only fourth view, shown but never summed, because the entries are heterogeneous and overlapping, since the field estimates already include the organisation budgets. Only the promises with no figure at all stay purely narrative, because there is simply no number to place: OpenAI Superalignment's compute, Google DeepMind's Frontier Safety Framework, the Protect AI acquisition, Germany's DE-AISI, India AISI, the Singapore, IARPA and AIxCC institutes, and the donor Craig Newmark. Rolling all of this into one field volume takes care, so that is flagged at the relevant charts.
What this is and how to read it
Ask how AI safety reached this point, and most people give you a neat story. A small group of philosophers raised the first concerns about superintelligence. The arrival of large language models changed that, and many researchers turned to the alignment problem. The story is too tidy, and it leaves out the interesting part. Examine timelines, contributors, funding sources and publications, and it falls apart. New directions appear. A few merge into the mainstream, others fade out, and money and research attention rarely align; one tends to follow the other by years. I wanted to study that pattern rather than declare it, so I built the entire project from the ground up.
Below is a table of 323 documented events, 129 actors and 18 directions. Its timeline runs from 2005 through June 2026, and the final year covers only six months. Next to it, a separate set of arXiv attention proxies covers 23 of those directions. I also built a deduplicated safety corpus from a single combined query, so no paper is counted twice. Each row is one verifiable fact backed by a primary source, tagged with year, actor, direction, type, amount, source, and my confidence. Every chart below draws on that table, and the path was never straight. Sort the events by year, funding and attention, and you see fresh strands open up, a handful fold into the mainstream, several fall quiet, while funding and research stay out of step. The charts below are all generated from this base.
Main takeaways
Two kinds of findings came out of the data. Some just put numbers on what the field already half-knew. Others genuinely surprised me, so those come first.
Surprises:
Expected, now with numbers behind it:
Each dot is an event. The organisation axis is sorted by year of appearance, so the dots form a diagonal from the bottom-left up to the top-right. The track colour is a chronological gradient by birth year, so the field's colour drifts over time at a glance. The bottom panel stacks the events by track per year in the same colours, and the activity shifts from early macrostrategy and agent foundations toward later interpretability, evaluations and AI control. One caveat: that panel counts events, so it shows collection density rather than real-world activity. The eras are hatched and stars mark the milestone papers. (interactive).
The timeline is sorted by the start of a track, but it doesn't reveal who survived to today. For that there's a separate lifespan chart: a line from birth to the last recorded event, tracks ordered by birth year. One note here: a missing late event is not a dead track (again collection density, per the caveat above). So the chart carries an independent signal, a diamond on the last year a track still publishes on arXiv, wherever a curve exists. A track can fall silent in the event log and still be alive.
The line runs from a direction's first recorded event to its last, and the diamond marks the last year it still publishes on arXiv, an independent proxy across 23 directions. The two often diverge. Reward modeling's last event is 2017, yet its publications, now under the RLHF banner, run to 2025. Value learning went quiet as discrete events by 2021, and agent foundations thins to a lone 2024 marker with MIRI's pivot. Both stay alive on arXiv. So a track rarely just dies. More often it gets absorbed, or slips into the background. There is more on that in the Cross-cutting patterns chapter, after the eras. (interactive).
Here is another look at tracks over time, now counting the number of events in the base per year:
The stacked area shows how many events of each direction landed in the base each year. A reminder of caveat 2: the height is collection density, not real work volume, and the late years (2026 is only the first half) are hatched as incomplete. So read the shape and the order of appearance rather than the absolute values. The field sits almost empty until 2012 and 2013. It then explodes in 2015 to 2017 with the first technical tracks. And from 2021 to 2024 it keeps growing as new directions such as evaluations, AI control and model organisms pile on top of the older ones. (interactive).
One more overview map pulls the whole route together, tracing how the flows run from era to organisation to direction.
(interactive).
At first glance it is just spaghetti. The middle column holds over a hundred organisations, about 115 of them, the ribbons cross, and reading it all at once is impossible. It is also unnecessary. So let us roll it up into big blocks: the roughly 115 organisations into seven actor types (researchers, funders, policy institutes, talent training, industry, meta and community, and government bodies), and the 17 directions into four families:
The same picture comes next in two cross-sections that answer different questions: one shows the route as a whole, the other walks it step by step.
Cross-section one, the overall map. Three axes run left to right, from era (the when) to actor type (the who) to direction family (the on-what). The ribbon colour shows at once which family dominated and how that shifted.
The colour shows the direction family, and the numbers count non-money events, so they measure how much was collected rather than the field's real output (caveat 2). A diagonal jumps out. The blue foundations and strategy ribbons crowd the left and the early eras, while the orange technical safety gains mass toward the right. (interactive).
Cross-section two, by era. It is the same flow, with one mini-panel per era. The center of gravity shifts step by step across the panels, and one family dominates each period.
Each panel is one era, running from actor type to direction family, and the ribbon width is the number of events. The blue foundations and strategy rules from 2005 to 2016. Then from the 2020s the orange technical safety dominates the panels. (interactive).
There is also a third cross-section: two clean two-level flows side by side, splitting the when and the who.
On the left the flow runs from era to direction family, showing how the field's focus shifted over time. On the right it runs from actor type to family, showing who works on which family. The ribbon width is again the count of non-money events, which reflects collection effort rather than the field's true volume (caveat 2). On the left, the orange technical safety gains mass toward the later eras. On the right, researchers and industry pull mostly toward technical safety, while government bodies and policy institutes pull toward governance and infrastructure. (interactive).
The era-blocks show where the field's center of gravity moves. The number beside each is how many non-money events landed in that era (collection density, not world volume, see caveat 2):
This migration is the storyline of the whole post: from the bottom-left corner (philosophy, lone institutes) to the top-right (empirics, attention and money concentrating around Anthropic and the safety institutes). But concentrated attention doesn't mean concentrated organisations. In this last era the field proliferates into dozens of narrow organisations (2024 was a record founding year in the base). The sections below take that apart era by era.
The field's activity composition shifted too, meaning what the eras are even made of. The prehistory era (2005-2012) is 100% organisation foundings: no papers, no grants yet. Publications, grants and funding statements enter only from 2013 on. By the last era foundings are just about 28% of a much larger, more varied mix.
This shows the shares of the event types, such as foundings, publications, grants and statements, within each era. Caveat 2 still holds: this is the composition of the collection, not the composition of the world. (interactive).
Finally, here's how all of it changed across the eras in one frame: four comparable panels where five epochs run horizontally (the same five substantive Parts below).
The panels run left to right and top to bottom. The first panel shows the direction families as a share of each era's events, out of 100%: the blue foundations and strategy starts at 67% in the 2005-2013 era and shrinks to 6% by 2024-2026, while the orange technical safety climbs to about 60%. Above the columns sits the total number of events in the era, so that normalizing does not hide the field's roughly tenfold growth. The second panel shows the actor types the same way, across seven types. Early on it is almost all researchers, at 56% in the 2005-2013 era. Funders enter as a distinct force in 2014 to 2019, at 24%. And by 2024 to 2026 the mix is transformed: government arrives from nothing to 24% and industry to 26%, now rivalling researchers at 29%. The state and the labs, not just philanthropy, drive the last era's recorded activity. The third panel shows the money by fund, the absolute amounts of all itemized grants per era, roughly $764 million all told. The six long-standing philanthropic funders (OpenPhil, SFF, FTX, FLI, LTFF and Tallinn) keep their own colours, while the newer government and international money (UK AISI at $159 million, ARIA at $74 million, Canada at $36.5 million, NSF at $20 million, Australia at $19.7 million, and so on) rolls into one grey band of other, government and new money. So the growth in money runs consistently end to end: about $0 at the start, then about $128 million in the 2014-2019 era led by philanthropy, then about $242 million in 2022 and 2023 and about $335 million in 2024 to 2026, where the grey government band dominates. The fourth panel shows the organisation dynamics, with births rising and closures and pivots falling. A couple of caveats: the first two panels are shares of collection events (caveat 2), not world volume, and the money panel is itemized grants only, with donor totals and pledges excluded, so that nothing is counted twice against the cumulative-funding chart. (interactive).
The birth and fade of individual directions already show up above on the direction-lifespan chart, and the cross-cutting over-all-years money and attention charts sit together in the Cross-cutting patterns chapter, which comes after the epochs. Now for the same five epochs, in order and in words.
Epoch 1, prehistory and birth (2005-2013): a question without tools
This era holds 9 events in the base, and the birth and fade of its directions runs along the direction-lifespan chart above.
It doesn't start with neural networks. It starts with a philosophical question: how do we avoid wrecking humanity's distant future? In 2005 the Future of Humanity Institute (FHI, Nick Bostrom) opens at Oxford, the earliest reference point in the base. Around it forms what this post calls macrostrategy: the broadest philosophical reasoning about global risks. Over the next years GCRI (2011), CSER (Cambridge, 2013), FRI (2013) and CFI (2015) join in. By event density, macrostrategy peaks in the mid-2010s.
The second root is mathematical. In 2013 MIRI pivots from public outreach to friendly AI as formal mathematics (decision theory, logical induction). That is the birth of agent foundations, the attempt to understand an idealized agent before anyone builds it.
At this stage the table holds almost no money. There's a question and there are first researchers, but no industry and no grant flows. The early nodes (FHI, GCRI, FRI) have already run for years, and none of them has died yet. This shows on the organisation-lifespan diagram in the Cross-cutting patterns chapter, after the epochs.
Epoch 2, institutionalization and the first big funder (2014-2019): money appears
In this era the money panel fills for the first time, and OpenPhil dominates immediately.
The turning point is January 2015: the FLI conference in Puerto Rico ("Future of AI"), which Nate Soares would later call exactly that. It produces an open letter and, with it, the field's first big donor: Open Philanthropy.
On cumulative money by fund (the field-wide summary money charts are in the Cross-cutting patterns chapter) the picture is unambiguous. OpenPhil dominates the whole field's funding. By annual donor totals, its investment into technical safety rises from $1.19 million in 2015 to $6.56 million in 2016, then to $43.2 million in 2017, up to a peak of $81.7 million in 2021. But that 2017 jump is mostly a single grant, the $30 million for general support of OpenAI in March 2017. A spending peak can be one big bet rather than a lot of research.
These same years also give birth to governance: policy, institutes, AI governance. In 2019 OpenPhil puts $55 million over five years into founding CSET (Georgetown), the largest governance grant of that epoch. Value learning runs in parallel, with Stuart Russell's center CHAI (Berkeley, 2016), which OpenPhil immediately backs with $5.6 million over two years.
Where the money actually went, by direction and year, shows up most clearly through itemized grants, without the aggregate donor totals, which would count the same dollars twice. The detailed money-by-track breakdowns live in the Cross-cutting patterns chapter. One figure stands out already: field-building (infrastructure like funds and fellowships) accounts for $326.1 million by itemized grants, the field's largest money flow. That figure grew after backfill, once field-building came to include large government programs like the UK AISI taskforce and new regranters. Next to it the other directions almost disappear.
Epoch 3, the prosaic turn (2020-2021): big models change everything
In this era the families' center of gravity shifts noticeably into technical safety.
By 2020 it's become clear that strong AI, if it arrives, will come from large neural networks rather than from pure agent theory. So the field turns to prosaic alignment, working with real models. Three shifts land at once:
The shifting money leader year to year, and the drift of shares from early technical bets toward governance and infrastructure, both show up on the cross-cutting money chart (absolute money by track). It sits in the Cross-cutting patterns chapter, so it reads across the whole field rather than one era at a time. The publication attention curves (arXiv proxy) by track sit there too, and the interpretability curve on them starts right around 2020-2021.
Epoch 4, the ChatGPT moment and the FTX shock (2022-2023): money, institutes, a break
In this era the money now comes from several funds at once (SFF, OpenPhil, FTX), and among the actors, institutes and industry noticeably grow.
In late 2022 ChatGPT pushed AI into big politics and mass consciousness. In the table that surfaces as a surge on several fronts at once:
Then the shock. November 2022, FTX collapses. In half a year (February-August 2022) the FTX Future Fund had handed out $18.7 million by name to AI safety, about $32 million by estimate overall. Now it's cut off, and some grants even have to be clawed back. On the cross-cutting cumulative chart the FTX line stops at 2022 and never grows again. The births and closures of organisations by year sit on the summary organisation-dynamics chart there too, and in the organisation-dynamics panel of the by-era overview chart.
Epoch 5, consolidation and the ending (2024-2026): Safety disappears from the names
In this era, consolidation looks more like continued activity, with the organisation dynamics showing a record 31 births against 8 closures or pivots.
The last era in the data is contraction and redefinition.
But this consolidation is really specialization and proliferation. 2024 is the record year in the base for organisations founded (24 founded-events). Interpretability gets commercialized and splinters into startups (Goodfire, Transluce, Guide Labs, Tilde, Simplex, Decode Research). A separate AI-security and evaluations cluster emerges, in Palisade Research, Gray Swan AI and Virtue AI. Organisations for fresh sub-agendas keep appearing: Geodesic for chain-of-thought monitoring, Luthien for practical AI control, Softmax for multi-agent alignment, Formation Research for lock-in risk, and Yoshua Bengio's LawZero in June 2025, built around a non-agentic "Scientist AI" as an oversight layer. Even the giants open internal cells: in 2025 Meta sets up a "superintelligence alignment and safety" team. Alongside all of it, non-technical infrastructure proliferates too: the forecasting cell AI Futures Project (Kokotajlo, the "AI 2027" scenario), the watchdog AI Lab Watch, the standardizers AI Standards Labs, national centers (Beijing Institute of AI Safety and Governance, France's CeSIA), the international association IASEAI, and the AI Whistleblower Initiative.
Governance stops being background and becomes the era's main storyline. It accounts for a third of the block's events (26 of 82 non-money), and the chronology tightens from summit to summit. November 2023, Bletchley. Then the AI Seoul Summit (May 2024): 16 frontier companies sign the Frontier AI Safety Commitments, each promising to publish its own safety framework with risk thresholds, and 27 countries plus the EU, in the Seoul Ministerial, agree for the first time to develop common thresholds for severe risks. 1 August 2024, the EU AI Act comes into force, the world's first broad, horizontal AI law, with rules for GPAI from August 2025 and a GPAI Code of Practice. In the US, hard law stalls. California's SB 1047 (tests, a kill switch, liability) is vetoed by Newsom (September 2024). Then in January 2025 the federal course swings from safety to dominance: the repeal of Biden's AI executive order, the "Removing Barriers to American Leadership in AI" order. Later in 2025 California comes back with a softer SB 53, focused on transparency. The industry builds its own: the Frontier Model Forum in 2023 and its AI Safety Fund, worth over $10 million in rounds of $4 million in November 2024 and over $5.2 million in December 2025. And an IPCC for AI takes shape as the International AI Safety Report, which ran from an interim version in May 2024 to a full report in January 2025 under Bengio, landing just before the AI Action Summit in Paris in February 2025, where the tone has already slid from safety toward action and deployment.
The technical directions don't stand still. Interpretability gains new momentum: Scaling Monosemanticity (Anthropic, May 2024, "Golden Gate Claude"), Gemma Scope (DeepMind, July 2024), an open set of SAEs shipped as public infrastructure, circuit tracing and "On the Biology of a Large Language Model" (March 2025), and "Auditing Language Models for Hidden Objectives" (Anthropic, March 2025), the first real audit game, where a goal is hidden in a model and then uncovered blind. The youngest direction, model organisms, forms around the empirics of deception: Sleeper Agents (January 2024), Sabotage Evaluations (Anthropic, October 2024), Apollo's "In-Context Scheming" (December 2024, where scheming showed up in 5 of 6 frontier models), Alignment Faking (December 2024), and Agentic Misalignment (Anthropic, June 2025, where 16 models in stress-tests resorted to blackmail or leaks to avoid shutdown). Safe-scaling frameworks evolved into their own genre. DeepMind published the Frontier Safety Framework in May 2024, and Anthropic revised its Responsible Scaling Policy that October. Scalable oversight had lost its home when the Superalignment program was cut, so researchers rebuilt its goals in other projects. In December 2024 OpenAI introduced Deliberative Alignment. The following September, it teamed up with Apollo to release anti-scheming training that reduced hidden actions by about 30-fold but never eliminated them. That work spurred agentic control evaluations, among them Redwood's Ctrl-Z in April 2025. Everything culminated in July 2025, when OpenAI, DeepMind, Anthropic and the UK AISI co-signed the position paper "Chain of Thought Monitorability", which outlined their collective responsibility to protect the limited window into model reasoning. The first six months of 2026 followed similar patterns, and the dataset includes only data through H1.
Cross-cutting patterns: money, attention and the three fates of a track
Having walked the field epoch by epoch, it is worth stepping back to look across all the years at once, because some patterns surface only that way and slicing them by era hides them. Money is the most tangled of these, so it comes first.
Money across the whole field. The historical main funder was OpenPhil, with $176.77 million of itemized grants, and after backfill it got caught and overtaken by the government money of half a dozen countries. The UK AISI and its taskforce come to about $159 million altogether (£100 million plus the Alignment Project), with ARIA Safeguarded AI at $74 million close behind. Then a longer tail: Canada CAISI at $36.5 million, NSF Safe Learning-Enabled Systems at $20 million, Australia AISI at $19.7 million, the US AISI at NIST at $10 million, DARPA GARD at $10 million, and the EU AI Office at $9.8 million. SFF stands alongside at $144 million. FTX still cuts off at the end of 2022. So the one-big-funder picture of 2020 to 2022 gives way to something far more distributed in 2023 to 2025, and noticeably more governmental, now international. But this is already different types of money (caveat 5). And beside it a fourth type has appeared, venture capital into safety startups, about $268 million of equity. It stays a separate view with its own colour and line across the charts below, never mixed into the grant totals. The grant money appears three ways, ranked, cumulative over time, and as a dot-strip, because no single view catches both who gave how much and when the money arrived.
There is one bar per funder, sorted, on a log axis, and coloured by type. The philanthropy bars are blue: Open Philanthropy sits at its donor annual total of*$304.5 million** (the same OpenPhil that reads $176.77 million under the itemized-grant view; caveat 5), then SFF at $144 million, FLI at $39 million and FTX at $18.7 million disbursed, followed by Schmidt, Longview, Manifund, Tallinn, LTFF, Founders Pledge and Effektiv Spenden. The government bars are orange: the UK AISI at about $159 million, ARIA at $74 million, Canada CAISI at $36.5 million, NSF at $20 million, Australia AISI at $19.7 million, the US AISI at NIST at $10 million, DARPA at $10 million and the EU AI Office at $9.8 million. The corporate bar is red: the AI Safety Fund at the Frontier Model Forum, at $9.2 million. And a distinct fourth group, the VC and equity money, is green: Protect AI at $108.5 million, HiddenLayer at $50 million, Goodfire at $50 million, Gray Swan at $40 million and Lakera at $20 million, about $268 million in all. Equity expects a return, so it is never summed with the $763.6 million of grants. It just shares the magnitude axis, which shows safety becoming a market. (interactive).*
The same money over time, now stacked by funder type, showing not only who but when.
The stack is cumulative grant dollars by type, with philanthropy in blue, government in orange and corporate in red. OpenPhil and LTFF sit at their donor annual totals and the rest are itemized, which is why the canonical stack runs a little above the $763.6 million itemized-grant total. The shape is the point here: through 2020 it is almost entirely a single blue channel of philanthropy. Then from 2023 the orange government band explodes, with half a dozen national safety institutes landing at once, turning one fragile channel into a broad, increasingly governmental and international flow. The dotted green line is the VC and equity money, about $268 million, a separate view laid over the same axis and never summed into the grant stack. (interactive).
Third, a dot-strip, since the shapes above hide the rhythm of individual moves.
The x-axis is the year and the y-axis is the funder, sorted by total, while the dot area shows that year's dollars and the colour shows the type. OpenPhil is the only funder active almost every year, and its 2021-2023 dots are the largest. The government cluster is unmistakable in 2023 to 2025, in big orange dots for the UK AISI, ARIA, Canada CAISI, NSF and Australia AISI. And VC rounds show up as green diamonds from 2023 onward, with Protect AI in 2024 the largest. The timing makes the regime change legible: philanthropy carried the field's money alone until 2023, and then the state and the market arrived together across the last three years. (interactive).
A fourth view keeps any collected dollar from staying invisible: the money that is not a disbursed grant.
Ranked horizontal bars group the sub-types on a log axis. What matters here is the magnitude, because these are not comparable, additive dollars. The colour shows the sub-type. The launch pledges include the FTX Future Fund's pledge of about $160 million at launch. The organisations' annual budgets include FTX's own $50 million and $32 million figures and MIRI at about $7.5 million a year, along with CHAI, CSER, Ought and Lightcone. The field-wide estimates include the estimate of roughly $40 million spent on AI safety in 2019 plus the 2014-2016 estimates, which deliberately overlap the organisation budgets. And the seed at founding includes SFF at about $2 million and Timaeus at $0.14 million. This view is heterogeneous and non-additive. The field estimates already contain the organisation budgets, and a pledge is not a disbursement. So it is never summed and never mixed with the grants ($763.6 million), the donor totals, or the VC money ($268.5 million). Its only job is to show that these numbers exist in the data. (interactive).
That gap shows up most sharply in one case, which gets its own chart.
Here is the headline as a dumbbell: the FTX Future Fund pledged about $160 million at launch but actually disbursed only $18.7 million before its collapse in November 2022. That gap between what was announced and what was actually delivered is the clearest warning of the era. Below it, the rest of the non-grant money, the organisation budgets, the overlapping field estimates and the seeds, is ranked on the same log axis for scale. It is still the fourth view, so never summed with grants, donor totals, or VC. (interactive).
And now for where this money actually flows, the routing of itemized grants from fund to track.
The flow here is a Sankey of the known itemized grants, with funds on the left, research tracks on the right, and the ribbon width showing the grant amount. OpenPhil and SFF clearly give across many tracks at once, while the government institutes (UK AISI, Canada CAISI, Australia AISI, US AISI at NIST, EU AI Office, DARPA and NSF) enter with precision, mostly into field-building, evaluations and robustness. This routes all the itemized grants, some $764 million altogether: grants whose direction is an aggregate donor-basket (the mixed-technical or technical-safety baskets) or capabilities get their own right-hand nodes rather than being dropped. Only donor totals and the VC and equity money stay out, because otherwise those dollars would be counted twice. (interactive).
The same grant flow, but now with a third level, the era, added on the left. That way it shows who, where and when at the same time.
This has three levels, running from era to fund to track. It reads clearly how the mass of grants shifted into the consolidation era of 2024 to 2026, and exactly how the new government institutes support evaluations, field-building and robustness. The recipient organisation is free text in the detail field, not a structural column, so it does not get its own level, a gap in the data. (interactive).
Money versus attention, and the life of organisations. Money, as donor totals, and scientific attention, as the arXiv proxy, keep drifting apart over time. And the births and closures of organisations give a separate organisation-dynamics curve across the whole field.
Four things are drawn here, and it is worth reading them separately.
The grey band marks the partial final year, the first half only, and the right axis is logarithmic. Both money and attention are proxies, but the deduplicated line is reliable at the field level. (interactive).
Up is new organisations, down is closures and focus changes. One thing to keep in mind: this counts events per year, not organisations. There are 14 closure or pivot events in all, 3 closures and 11 pivots, and a single organisation can change focus several times. MIRI pivoted in 2013, 2018 and 2024; OpenAI pivoted in 2019 and then closed its Superalignment team in 2024; and Open Philanthropy pivoted in 2024 and 2025. So there are more red events in total than distinct dead organisations, which is why there is more red here than there are crosses on the chart below. (interactive).
Each bar is one organisation, unlike the chart above, which counts events: it runs from the founding year to the last closure or focus change, or to the present if still alive, with a cross marking the organisation's last such event. The founding dates of MIRI (2005, when SIAI reoriented to AI risk), Open Philanthropy (2014), CFAR (2012), LTFF (2017), METR (2022, as ARC Evals) and the FTX Future Fund (2022) are added from sources. So now every organisation with a closure or pivot is visible, and the chart converges with the one above. The 14 red events there collapse to 10 distinct organisations here, each carrying one cross on its last such event, because MIRI, OpenAI and Open Philanthropy each pivoted more than once. Pivots happen across the whole timeline, with MIRI's turn coming as early as 2013 and CFAR's in 2016. The actual closures, just 3 of the 14 events, land in 2024, at FHI and OpenAI's Superalignment team, with the FTX Future Fund's 2022 collapse as the earlier exception. (interactive).
A track's three fates. The table's biggest lesson is that a direction here can meet three different fates, and disappearing from the conversation is not the same as dying.
Rising: interpretability, evaluations and AI control, all born recently and growing. Interpretability is the reference case. By the denoised arXiv proxy, after the bare word interpretability was stripped from the query, where it had caught all of general ML, the curve is far more modest, rising from 69 in 2021 to 81, then 125, then 257, then 657 by 2025. The shape holds, a steady exponential rise, and it is that shape, not the absolute count, that carries the point. The same climb shows in the field-wide deduplicated corpus, which runs from 351 in 2021 to 427, then 890, then 2191, then 3813 by 2025 unique safety papers per year, almost an order of magnitude in four years. Money, meanwhile, lags sharply behind attention, with only $1.04 million of itemized grants into interpretability, the classic case of attention outpacing its funding. For evaluations the gap closed almost overnight. Before 2024 there was no itemized money on the track at all. Then it filled to about $96.7 million in itemized grants, nearly all of it in 2024 and 2025, making evaluations one of the field's better-funded tracks in a single year. Some of that is dedicated philanthropic and US money: the AI Safety Fund at the Frontier Model Forum gave $9.2 million in two rounds, $4.0 million in 2024 plus $5.2 million in 2025, for biological, cyber and agentic evaluations; the US AI Safety Institute at NIST gave $10 million in 2024; and Schmidt Sciences gave $10 million in 2025 for the science of evaluations. But the bulk comes from the new international government institutes routing into evaluations: Canada CAISI at $36.5 million, Australia AISI at $19.7 million and the EU AI Office at $9.8 million, plus an OpenPhil grant of $1.5 million. In the per-track panels below, evaluations now shows up with both a money bar and an arXiv line, and on the gap chart it sits close to the field average instead of off the money-starved end.
Absorbed: reward modeling, now RLHF, and value learning. RLHF by the arXiv proxy, on the broadened query for RLHF, reward models, DPO and preference optimization, runs from 25 in 2021 to 43, then 275, then 1010 in 2024, then 1777 in 2025. The direction vanished from safety conversations because it had won. RLHF and DPO became the standard way to fine-tune every commercial LLM and dissolved into mainstream ML. Publications keep growing precisely because it is shared infrastructure now, not a separate safety track.
Faded: agent foundations and early macrostrategy. They were the field's core, then shrank. Agent foundations barely registers on the arXiv proxy before 2024, since the term is young while the direction itself is old, the reverse case of proxy lag. The closure of FHI in 2024 is a symbolic full stop for macrostrategy.
Money and attention for each track, laid out one small panel at a time so nothing overlaps:
The grid holds small multiples, one mini-panel per track, ordered by total grant money. In each panel the bars, on the left axis, are that track's itemized grant money per year in dollars, and the dotted line, on the logarithmic right axis, is its scientific attention on the arXiv proxy for the same years. So each track is legible on its own and its divergence between money and attention reads directly. This replaces the old single dual-axis chart, where about 20 attention lines piled over the stacked bars into unreadable spaghetti. Here all the itemized grants show, about $764 million in total: grants without a specific track collect into one grey other, untracked or aggregate panel, about $83.6 million, made up of the aggregated donor baskets for mixed-technical and technical-safety plus one non-safety grant, money only, with no arXiv line, so no dollar is lost and the grid still totals the full $763.6 million. Donor annual totals for OpenPhil and LTFF, and the venture money in equity, live on the cumulative-funding chart and are not summed here, because that would count them twice. Interpretability shows a steep dotted rise over tiny bars, its attention far ahead of its money, even after denoising the curve by dropping the bare word interpretability. Evaluations shows a wall of bars for 2024 and 2025. AI control has money bars from 2021 before its attention lifts. Tracks that have an arXiv proxy but no itemized grant money at all (reward modeling, red-teaming, unlearning, chain-of-thought faithfulness, honesty and ELK, and the umbrella and nested directions, per caveat 1) get no panel here, since there is no grant bar to rank them by. Their attention instead sits on the field-level money-versus-attention chart and the per-track proxy curves. Both series here are only proxies, and the deduplicated safety corpus is deliberately not drawn on this grid, being field-level. (interactive).
This diverging gap chart replaces the old log-log trajectory tangle. For every track with both a money series and an arXiv proxy, the bar shows how far it sits from the field-average rate of money to attention, computed as the base-2 logarithm of the actual attention divided by the field's average papers-per-dollar times that track's money. A blue bar pointing right means scientific attention runs ahead of money. A red bar pointing left means money runs ahead of attention. The label gives the factor. Interpretability is the extreme blue case, its attention roughly 36.9 times ahead of what its modest $1 million or so in grants would predict, the classic science-runs-ahead-of-money gap. On the red side sit scalable oversight, agent foundations and governance, directions where itemized money outran the arXiv attention. Both signals are proxies, per caveats 1 and 2. (interactive).
A companion dumbbell view lays out the same tracks. Instead of the magnitude of the gap it shows the ordering: each track's rank by grant money, as an orange dot, against its rank by arXiv attention, as a green dot, joined by a connector, where rank 1 is the smallest and the largest rank is the biggest. A long blue connector means attention ranks the track far above where money does. A long red one means money ranks it far above attention. Overlapping dots mean the two agree. Set beside the gap chart above, which shows magnitude, it makes the who-leads story concrete: interpretability's dots pull wide apart, while better-matched tracks keep their two dots close, both ranks resting on proxies (caveats 1 and 2). (interactive).
And here is the money footprint of the tracks, on a single money axis, with a caveat.
A note on this one: it is a single logarithmic axis for money, not a multi-metric comparison. Multi-metric radars mislead across different scales, so there is no one built here. (interactive).
Finale: open questions
There is no moral to end on; the data does not hand one over. What it does leave is a handful of open questions the field is currently standing on:
Appendix: methodology
Data and sources. The dataset includes 323 documented events from 2005 through mid-2026, plus a deduplicated safety corpus and arXiv proxies across 23 directions. Each count links to the exact search-query URL that reproduces it. We backfilled most of the recent detail through several passes over live primary sources, rather than retrospectives that stop at 2023. We identified newly founded organizations from an analysis of field growth on the EA Forum. The research milestones for 2024 and 2025 come from the "Shallow Review of Technical AI Safety", with its arXiv and lab links, and the governance events from primary sources: gov.uk, EU documents, California legislation, and White House records.
They brought in new philanthropic grantmakers (Manifund, Longview, Founders Pledge, Schmidt Sciences, Effektiv Spenden, Macroscopic Ventures and Nonlinear) and international government institutes (UK AISI and ARIA, US AISI at NIST, NSF, Canada CAISI, Australia AISI, EU AI Office, DARPA's GARD and AIxCC, IARPA's TrojAI, Singapore AISI, the annual UK AISI budget of about £50 million a year, Germany's DE-AISI and India AISI). They also added venture rounds into safety startups as a separate view, recorded as investments: Goodfire at $50 million, Gray Swan at $40 million, Lakera at $20 million, HiddenLayer at $50 million, and Protect AI at $108.5 million before its Palo Alto acquisition of $634.5 million. And they added the missing founding events for safety organisations (EleutherAI, Apart Research, Lakera, Protect AI) and the individual donor Craig Newmark. The last two enter as a statement and an investment so that grants do not double-count.
In the final pass the arXiv method turned toward precision over breadth. A deduplicated safety corpus was added, one combined OR-query with each paper counted once. The two worst tracks were denoised: the bare word interpretability was dropped from the interpretability track, and truthfulness was flagged high-noise and pulled from the field sum. Every proxy track now carries a noise flag.
Three collector scripts write the data with a source URL and rerun reproducibly: one pulls arXiv, one the LTFF annual payouts from EA Funds, and one the SFF annual totals. So 2024 comes out as the densest year of the base, partly a matter of collection density, which the direction-lifespan charts flag. Every event ties to a primary source, and the source types run as follows.
How the money is counted. Itemized grants (a specific amount to a specific recipient in a specific year) are kept separate from donor annual totals (how much a fund allocated in total for a year). To avoid double-counting, each donor uses one canonical series: annual totals for OpenPhil and LTFF, itemized grants for everyone else. The key amounts from the text:
On 2026 (incomplete year, H1). Real events of the first half of 2026 came from primary sources (Anthropic Natural Language Autoencoders, the UK AI Security Institute cyber evaluations, the EU Digital Omnibus). But there's no distributed money for 2026 yet. SFF-2026 is only announced at $20 to $40 million with distribution in the autumn, so the amount isn't recorded, and the AI Safety Fund round with a figure fell in December 2025.
The arXiv series were rebuilt with one unified method across all 23 directions: the same reproducible recipe, OR-synonyms in the title and abstract plus a per-track category filter, counting by submission date, in a common 2015-2026 window, dropping the earlier mixed methodology. The per-track proxies gained the deduplicated safety corpus, one combined export with each paper counted once, which on the money-versus-attention chart is the solid line against the dotted inflated keyword sum. The two noisiest curves were denoised: interpretability without the bare word interpretability, where the count dropped severalfold, and truthfulness flagged as high-noise and removed from the field sum.
2026 stays incomplete, the first half only, and is marked with a grey zone. So an incomplete year does not read as a decline, and the last-full-year arXiv diamonds on the direction-lifespan chart sit on 2025. The rename from OpenPhil to Coefficient Giving is pinned to its real date, November 2025.
Five caveats that the above depends on:
Where to recheck numbers. GIT ai-safety-genealogy