Yesterday I obtained Anthropic's[1] Certificate of Incorporation, and its past versions, from the State of Delaware. I don't recommend reading it.[2] This post is about what the CoI tells us about Anthropic's Long-Term Benefit Trust (context: Maybe Anthropic's Long-Term Benefit Trust is powerless).

Tl;dr: the only new information of moderate importance is the voting thresholds necessary to modify Trust stuff. My concerns all still stand in some form. Absence of badness is a small positive update.


Anthropic has vaguely described stockholders' power over the Trust:

a series of "failsafe" provisions . . . allow changes to the Trust and its powers without the consent of the Trustees if sufficiently large supermajorities of the stockholders agree. The required supermajorities increase as the Trust’s power phases in

The CoI has details: amending the CoI to modify the Trust requires a vote reaching the "Transfer Approval Threshold," defined as:

(1) prior to the date that is the one-year anniversary of the Final Phase-In Date [note: "the Final Phase-In Date" is in November 2024], either (a)(i) a majority of the Voting Common Stock then-outstanding and held by the Founders (as defined in the Voting Agreement), (ii) a majority of the Series A Preferred Stock then-outstanding and (iii) a majority of the voting power of the outstanding Preferred Stock entitled to vote generally (which for the avoidance of doubt shall exclude the Non-Voting Preferred Stock), but excluding the Series A Preferred Stock or (b) at least seventy-five percent (75%) of the voting power of the then-outstanding shares of the Corporation's capital stock entitled to vote generally (which for the avoidance of doubt shall exclude the Non-Voting Preferred Stock and any voting power attributable to the Class T Common Stock) and

(2) on and following the date that is the one-year anniversary of the Final Phase-In Date, either (x)(i) at least seventy-five percent (75%) of the Voting Common Stock then outstanding and held by the Founders (as defined in the Voting Agreement), (ii) at least at least fifty percent (50%) of the Series A Preferred Stock then-outstanding and (iii) at least seventy-five percent (75%) of the voting power of the outstanding Preferred Stock entitled to vote generally (which for the avoidance of doubt shall exclude the Non-Voting Preferred Stock), but excluding the Series A Preferred Stock or (y) at least eighty-five [percent] (85%) of the voting power of the then-outstanding shares of the Corporation's capital stock entitled to vote generally (which for the avoidance of doubt shall exclude the Non-Voting Preferred Stock and any voting power attributable to the Class T Common Stock)

If Anthropic's description above is about this, it's odd and misleading. Perhaps Anthropic's description is about the Trust Agreement, not just the CoI.

Per Article IX,[3] amending the CoI to modify the Trust also requires at least 75% of the board. This will apparently give the Trust tons of independence after it elects 3/5 of the board! Or at least, it will give the Trust tons of protection from CoI amendments — but not necessarily from Trust Agreement shenanigans; see below.


Before reading the CoI, I had 4 main questions/concerns about the Trust:[4]

  1. Morley et al.: "the Trust Agreement also authorizes the Trust to be enforced by the company and by groups of the company’s stockholders who have held a sufficient percentage of the company’s equity for a sufficient period of time," rather than the Trustees.
    1. I don't really know what this means. And it's vague. It sounds like a straightforward way for Anthropic/stockholders to subvert the Trust.
  2. Morley et al.: the Trust and its powers can be amended "by a supermajority of stockholders. . . . [This] operates as a kind of failsafe against the actions of the Voting Trustees and safeguards the interests of stockholders." Anthropic: "the Trust and its powers [can be changed] without the consent of the Trustees if sufficiently large supermajorities of the stockholders agree."
    1. It's impossible to assess this "failsafe" without knowing the thresholds for these "supermajorities" and ideally who controls voting shares and how that can change.
  3. Maybe there are issues with the CoI or Trust Agreement that we don't know about.
  4. The plan was for the Trust to elect a board member in September 2023. The Trust's election of Jay Kreps was announced in May 2024. Why the delay? (And how was he selected?)

Now:

  1. No change; we still can't see the Trust Agreement.
  2. We see the thresholds, quoted above. It's pretty low until November 2025, and after that I don't know — it depends on who controls various kinds of stock and how that can change.
  3. I didn't find major issues in the CoI, but someone with expertise in corporate law should read it too. No change on the Trust Agreement; we still can't see it.
  4. No change.

So: the Trust Agreement still has crucial answers. Who controls various kinds of stock is a crucial open question. (And how Anthropic can create or transfer voting power may also be an important question.[5]) (And there's the question around the election of Kreps.)

Given all this, most of Anthropic's/stockholders' power over the Trust (in expectation) will come from Trust Agreement stuff—and in particular Anthropic's/stockholders' power to "enforce" the Trust—which we can't see.

Note: I don't have the background to red-team the CoI well; if there was an obfuscated loophole, I probably wouldn't notice it.


I think the CoI is all standard corporate stuff except:

  • III (p. 1): "The specific public benefit that the Corporation will promote is to responsibly develop and maintain advanced AI for the long term benefit of humanity." We already knew this.
  • IV(D) (pp. 16–20): Class T Stock stuff.
    • Section (4) defines the "Transfer Approval Threshold," which is mostly important because of Article IX.
    • Sections (5)–(6) are various protections; I lack context to evaluate if they're exhaustive/sufficient but they seem good and consistent with Anthropic actually trying to give the Trust some effective independence.
  • VIII(B) (pp. 20–22): board composition. We already knew that the Trust will have the power to elect 2/5 board members in July and 3/5 in November. The other two board seats will be controlled by the "Electing Preferred Holders" and "the holders of record of the Voting Common Stock"; I don't know who that is.
  • XI (p. 24): amending the CoI. Changing IV(D) or XI requires reaching the Transfer Approval Threshold and getting at least 75% of the board.[6]

Crossposted from AI Lab Watch. Subscribe on Substack.

  1. ^

     Why not other AI companies? I had questions about Anthropic's Long-Term Benefit Trust; I don't have analogous questions about other companies. One analogous OpenAI document has been published by Vox. I might look for random information on OpenAI, but OpenAI is more complicated. And DeepMind isn't incorporated in Delaware, and it's more complicated than Anthropic due to its relationship with Google.

  2. ^

     If you do read it, see the last section of this post.

  3. ^

     I think there's a tension between Article IV(D)(5)(a) and Article IX. I suspect I misunderstand Article IV(D)(5)(a) in the context of the whole document; I will ignore it here for now.

    My reading, which seems crazy: per Article IV(D)(4)(b), the Class T stock will exist perpetually unless its holder—the Trust—decides to convert it. Per Article IV(D)(5)(a), as long as the Class T stock exists, Anthropic can't disempower its holder.

  4. ^
  5. ^

     Article IV(D)(5)(b)(i)(3)(a) requires Anthropic to give the Trust 5 calendar days to intervene before selling or transferring stock with 50% or more of the total voting power, starting in July. I lack context to evaluate how well this or other protections suffice to protect the Trust from Anthropic taking over. I think I wish this said 2% rather than 50%, but I'm ignorant about the magnitude of the cost of that policy.

  6. ^

     I initially thought there was a loophole where Anthropic could amend VIII(B) to disempower the Trust, but I think IV(D)(5)(a) prevents this.

New Comment
4 comments, sorted by Click to highlight new comments since:

Looking into stuff and actually reading public information about AI companies seems very valuable and needed to me.

Quick updates:

  • I worried this was a loophole: "the Trust Agreement also authorizes the Trust to be enforced by the company and by groups of the company’s stockholders who have held a sufficient percentage of the company’s equity for a sufficient period of time." An independent person told me it's a normal Delaware law thing and it's only relevant if the Trust breaks the rules. Yay! This is good news, but I haven't verified it and I'm still somewhat suspicious (but this is mostly on me, not Anthropic, to figure out).
  • I started trying to deduce stuff about who holds how many shares. I think this isn't really doable from public information, e.g. some Series C shares have voting power and some don't and I don't know how to tell which investors have which. But I got a better understanding of this stuff and I now tentatively think the Transfer Approval Threshold is super reasonable. Yay!
    • Also even if I understood the VC investments perfectly I think I'd be confused by the (much larger?) investments by Amazon and Google
    • If you're savvy in this and want to help, let me know and I'll share my rough notes

(On the other hand, it's pretty concerning that the Trust hasn't replaced Jason, who left in December, nor Paul, who left in April. Also note that it hasn't elected a board member to fill Luke's seat.)

I guess my LTBT-related asks for Anthropic are now:

  • Commit that you'll promptly announce publicly if certain big LTBT-related changes happen
  • Publish the Trust Agreement (and any other relevant documents), or at least let a non-Anthropic person I trust read it (under an NDA if you want) and publish whether they have big concerns

(This isn't high-priority for me; I just get annoyed when Anthropic brags about its LTBT without having justified that it's great.)

The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

I'm still confused about Article IV(D)(5)(a) (p. 18) of the CoI. See footnote 3.