Looking into stuff and actually reading public information about AI companies seems very valuable and needed to me.
Quick updates:
(On the other hand, it's pretty concerning that the Trust hasn't replaced Jason, who left in December, nor Paul, who left in April. Also note that it hasn't elected a board member to fill Luke's seat.)
I guess my LTBT-related asks for Anthropic are now:
(This isn't high-priority for me; I just get annoyed when Anthropic brags about its LTBT without having justified that it's great.)
The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.
Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?
Yesterday I obtained Anthropic's[1] Certificate of Incorporation, and its past versions, from the State of Delaware. I don't recommend reading it.[2] This post is about what the CoI tells us about Anthropic's Long-Term Benefit Trust (context: Maybe Anthropic's Long-Term Benefit Trust is powerless).
Tl;dr: the only new information of moderate importance is the voting thresholds necessary to modify Trust stuff. My concerns all still stand in some form. Absence of badness is a small positive update.
Anthropic has vaguely described stockholders' power over the Trust:
The CoI has details: amending the CoI to modify the Trust requires a vote reaching the "Transfer Approval Threshold," defined as:
If Anthropic's description above is about this, it's odd and misleading. Perhaps Anthropic's description is about the Trust Agreement, not just the CoI.
Per Article IX,[3] amending the CoI to modify the Trust also requires at least 75% of the board. This will apparently give the Trust tons of independence after it elects 3/5 of the board! Or at least, it will give the Trust tons of protection from CoI amendments — but not necessarily from Trust Agreement shenanigans; see below.
Before reading the CoI, I had 4 main questions/concerns about the Trust:[4]
Now:
So: the Trust Agreement still has crucial answers. Who controls various kinds of stock is a crucial open question. (And how Anthropic can create or transfer voting power may also be an important question.[5]) (And there's the question around the election of Kreps.)
Given all this, most of Anthropic's/stockholders' power over the Trust (in expectation) will come from Trust Agreement stuff—and in particular Anthropic's/stockholders' power to "enforce" the Trust—which we can't see.
Note: I don't have the background to red-team the CoI well; if there was an obfuscated loophole, I probably wouldn't notice it.
I think the CoI is all standard corporate stuff except:
Crossposted from AI Lab Watch. Subscribe on Substack.
Why not other AI companies? I had questions about Anthropic's Long-Term Benefit Trust; I don't have analogous questions about other companies. One analogous OpenAI document has been published by Vox. I might look for random information on OpenAI, but OpenAI is more complicated. And DeepMind isn't incorporated in Delaware, and it's more complicated than Anthropic due to its relationship with Google.
If you do read it, see the last section of this post.
I think there's a tension between Article IV(D)(5)(a) and Article IX. I suspect I misunderstand Article IV(D)(5)(a) in the context of the whole document; I will ignore it here for now.
My reading, which seems crazy: per Article IV(D)(4)(b), the Class T stock will exist perpetually unless its holder—the Trust—decides to convert it. Per Article IV(D)(5)(a), as long as the Class T stock exists, Anthropic can't disempower its holder.
Adapted from Maybe Anthropic's Long-Term Benefit Trust is powerless. Note updates in the comments.
Article IV(D)(5)(b)(i)(3)(a) requires Anthropic to give the Trust 5 calendar days to intervene before selling or transferring stock with 50% or more of the total voting power, starting in July. I lack context to evaluate how well this or other protections suffice to protect the Trust from Anthropic taking over. I think I wish this said 2% rather than 50%, but I'm ignorant about the magnitude of the cost of that policy.
I initially thought there was a loophole where Anthropic could amend VIII(B) to disempower the Trust, but I think IV(D)(5)(a) prevents this.