x

LESSWRONG
LW

Tamsin Leake — LessWrong

Tamsin Leake

Top postsTop post

Tamsin Leake

Message

i unlisted some of my posts because of wide concerns about people being careless about exfohazards.

pgp key, email, proof, matrix: @carado4:matrix.org.

2923

Ω

71

8

206

5y

Tamsin Leake

i unlisted some of my posts because of wide concerns about people being careless about exfohazards.

pgp key, email, proof, matrix: @carado4:matrix.org.

Top postsTop post

Orthogonal: A new agent foundations alignment organization

We are putting together Orthogonal, a non-profit alignment research organization focused on agent foundations, based in Europe. We are pursuing the formal alignment flavor of agent foundations in order to solve alignment in a manner which would scale to superintelligence in order to robustly overcome AI risk. If we can afford to, we also intend to hire agent foundations researchers which, while not directly aimed at such an agenda, produce output which is likely to be instrumental to it, such as finding useful "true names". Within this framework, our foremost agenda for the moment is QACI, and we expect to make significant progress on ambitious alignment within short timelines (months to years) and produce a bunch of dignity in the face of high existential risk. Our goal is to be the kind of object-level research which cyborgism would want to accelerate. And when other AI organizations attempt to "buy time" by restraining their AI systems, we intend to be the research that this time is being bought for. We intend to exercise significant caution with regards to AI capability exfohazards: Conjecture's policy document offers a sensible precedent for handling matters of internal sharing, and unlisted posts are a reasonable default for publishing our content to the outside. Furthermore, we would like to communicate about research and strategy with MIRI, whose model of AI risk we largely share and who we percieve to have the most experience with non-independent agent foundations research. Including myself — Tamsin Leake, founder of Orthogonal and LTFF-funded AI alignment researcher — we have several promising researchers intending to work fulltime, and several more who are considering that option. I expect that we will find more researchers excited to join our efforts in solving ambitious alignment. If you are interested in such a position, we encourage you to get acquainted with our research agenda — provided we get adequate funding, we hope to run a fellowship whe

217Apr 19, 2023

We're all in this together

Orthogonal's Formal-Goal Alignment theory of change

How LDT helps reduce the AI arms race

Epistemic states as a potential benign prior

Malignancy in the prior seems like a strong crux of the goal-design part of alignment to me. Whether your prior is going to be used to model: * processes in the multiverse containing the AI which does said modeling, * processes which would output all of some blog so we...

Aug 31, 2024•31

How LDT helps reduce the AI arms race

(Epistemic status: I think this is right?) Alice is the CEO of ArmaggAI, and Bob is the CEO of BigModelsAI, two major AI capabilities organizations. They're racing to be the first to build a superintelligence aligned to their respective CEV which would take over the universe and satisfy their values....

Dec 10, 2023•65

We're all in this together

There's one thing history seems to have been trying to teach us: that the contents of the future are determined by power, economics, politics, and other conflict-theoritic matters. Turns out, nope! Almost all of what the future contains is determined by which of the two following engineering problems is solved...

Dec 5, 2023•69

formalizing the QACI alignment formal-goal

this work was done by Tamsin Leake and Julia Persson at Orthogonal. thanks to mesaoptimizer for his help putting together this post. what does the QACI plan for formal-goal alignment actually look like when formalized as math? in this post, we'll be presenting our current formalization, which we believe has...

Jun 10, 2023•54

Orthogonal's Formal-Goal Alignment theory of change

We recently announced [Orthogonal, an agent foundations alignment research organization. In this post, I give a thorough explanation of the formal-goal alignment framework, the motivation behind it, and the theory of change it fits in. The overall shape of what we're doing is: * Building a formal goal which would...

May 5, 2023•69

Orthogonal: A new agent foundations alignment organization

We are putting together Orthogonal, a non-profit alignment research organization focused on agent foundations, based in Europe. We are pursuing the formal alignment flavor of agent foundations in order to solve alignment in a manner which would scale to superintelligence in order to robustly overcome AI risk. If we can...

Apr 19, 2023•217