scottviteri

Very interesting! I'm excited to read your post.

Replying to«Boundaries», Part 3a: Defining boundaries as directed Markov blankets

«Boundaries», Part 3a: Defining boundaries as directed Markov blankets

I take back the part about pi and update determining the causal structure, because many causal diagrams are constant with the same poly diagram

Replying toThe Geometric Expectation

I think what is going on here is that both $\nabla^{*}$ and $G$ are of the form $(e^{\land}) \circ g \circ ln$ with $g = \nabla$ and $g = E$ , respectively. Let's define the star operator as $g^{*} = (e^{\land}) \circ g \circ ln$ . Then $(f \circ g)^{*} = (e^{\land}) \circ (f \circ g) \circ ln = (e^{\land}) \circ f \circ ln \circ (e^{\land}) \circ g \circ ln = f^{*} \circ g^{*}$ , by associativity of function composition. Further, if $f$ and $g$ commute, then so do $f^{*}$ and $g^{*}$ : $g^{*} \circ f^{*} = (g \circ f)^{*} = (f \circ g)^{*} = f^{*} \circ g^{*} .$

So the commutativity of the geometric expectation and derivative fall directly out of their representation as $E^{*}$ and $\nabla^{*}$ , respectively, by commutativity of $E$ and $\nabla$ , as long as they are over different variables.

We can also derive what happens when the expectation and gradient are over the same variables: $(\nabla_{θ} \circ E_{x \sim P_{θ} (x)})^{*}$ . First, notice that $(* k)^{*} (x) = e^{k * ln x} = e^{ln x * k} = x^{k}$ , so $(* k)^{*} = (^{\land} k)$ .. Also $(+ k)^{*} (x) = e^{k + ln (x)} = e^{k} e^{ln (x)} = x e^{k} ⟹ (+ k)^{*} = (* e^{k})$ .

Now let's expand the composition of the gradient and expectation. $(\nabla_{θ} \circ E_{x \sim P_{θ} (x)}) (f (x)) = \nabla_{θ} \int P_{θ} (x) f (x) d x = E_{x \sim P_{θ} (x)} [\nabla_{θ} (f (x) ln P_{θ} (x))]$ , using the log-derivative trick. So $\nabla_{θ} \circ E_{x \sim P_{θ} (x)} = E_{x \sim P_{θ} (x)} \circ \nabla_{θ} \circ (* ln P_{θ} (x))$ .

Therefore, $\nabla_{θ}^{*} \circ G_{x \sim P_{θ} (x)} = (\nabla_{θ} \circ E_{x \sim P_{θ} (x)})^{*}$ $= E_{x \sim P_{θ} (x)}^{*} \circ \nabla_{θ}^{*} \circ (* ln P_{θ} (x))^{*}$ $= G_{x \sim P_{θ}} \circ \nabla_{θ}^{*} \circ (^{\land} ln P_{θ})$ .

Writing it out, we have $\nabla_{θ}^{*} G_{x \sim P_{θ} (x)} [f (x)] = G_{x \sim P_{θ} (x)} [\nabla_{θ}^{*} (f (x)^{ln P_{θ} (x)}]$ .

Replying toThe Geometric Expectation

And if I pushed around symbols correctly, the geometric derivative can be pulled inside of a geometric expectation () similarly to how an additive derivative can be pulled inside an additive expectation ( $\nabla_{θ} E_{x \sim P (x)} [f_{θ} (x)] = E_{x \sim P (x)} [\nabla_{θ} f_{θ} (x)]$ ). Also, just as additive expectation distributes over addition ( $E [f (x) + g (x)] = E [f (x)] + E [g (x)]$ ), geometric expectation distributes over multiplication ( $G [f (x) g (x)] = G [f (x)] G [g (x)]$ ).

Replying to«Boundaries», Part 3a: Defining boundaries as directed Markov blankets

«Boundaries», Part 3a: Defining boundaries as directed Markov blankets

If I try to use this framework to express two agents communicating, I get an image with a V1, A1, P1, V2, A2, and P2, with cross arrows from A1 to P2 and A2 to P1. This admits many ways to get a roundtrip message. We could have A1 -> P2 -> A2 -> P2 directly, or A1 -> P2 -> V2 -> A2 -> P1, or many cycles among P2, V2, and A2 before P1 receives a message. But in none of these could I hope to get a response in one time step the way I would if both agents simultaneously took an action, and then simultaneously read from their... (read more)

Replying toThe Geometric Expectation

Actually maybe this family is more relevant:
https://en.wikipedia.org/wiki/Generalized_mean, where the geometric mean is the limit as we approach zero.

Replying toThe Geometric Expectation

Causality and a Cost Semantics for Neural Networks

The "harmonic integral" would be the inverse of integral of the inverse of a function -- https://math.stackexchange.com/questions/2408012/harmonic-integral

Democratic AI Constitution: Round-Robin Debate and Synthesis

$Epistemic status:$ I time-boxed this idea to three days of effort. So any calculations are pretty sloppy, and I haven't looked into any related works. I probably could have done much better if I knew anything about circuit complexity. There are some TODOs and an unfinished last section -- if you are interested in this content and want to pick up where I have left off I'll gladly add you as a collaborator to this post.

Here is a "tech tree" for neural networks. I conjecture (based on admittedly few experiments) that the simplest implementation of any node in this tree includes an implementation of its parents, given that we are writing programs starting from the primitives +, *, and relu. An especially surprising relationship (to me) is that "if statements" are best implemented downstream of division.

Introduction

While discussing with my friend Anthony Corso, an intriguing idea arose. Maybe we can define whether program $p_{1}$ "causes" $p_{2}$ in the following way: Given a neural network that mimics $p_{1}$ , how easy is it to learn a neural network which mimics the behavior of $p_{2}$ ? This proposition is intriguing because it frames causality as a question about two arbitrary programs, and reduces it to a problem of program complexity.

Suppose that $p_{1}$ and $p_{2}$ are written in a programming language P, and let P(ops) represent P extended with ops as primitive operations. We define a complexity function $C : P (o p s) \to R$ , which takes a program in the extended language and returns a real number representative of the program's complexity for some fixed notion of complexity. Let's define the degree to which $p_{1}$ "causes" $p_{2}$ as the minimum complexity achievable by a program p from $P (p_{1})$ such that p is extensionally equal (equal for all inputs) to $p_{2}$ . If $P_{2}$ is the set of all p in $P (o b s + p 1)$ that are extensionally equal to $p_{2}$ , then $c a u s e s (p_{1}, p_{2}) = {min}_{p \in P_{2}} C (p)$ . We can also use this definition in the approximate case, considering the minimum complexity achievable by programs p such that $E (p (x) - p_{2} (x))^{2} < ε$ with respect to some $L_{1} - i n t e g r a b l e$ probability measure.

We can define a particular complexity function $C$ that represents the cost of executing a program. We can estimate this quantity by looking at the program's Abstract Syntax Tree (AST) in rela...

This document was my initial response to OpenAI's call for proposals on how to democratically steer AI. I was thinking about crowdsourcing a constitution in the Constitutional AI sense. The idea is that:

Voters each propose an AI constitution
GPT selects a pair of constitutions, simulates a debate between the two ideas, and merges them into a single constitution that both parties would agree with
Repeat until there is only one constitution left

Below we are starting with demographic groups and such -- please mentally substitute them for individual voters. GPT doesn't do much here besides a union of ideas, but the hope is that this will change will more rounds of debate.

I ended up... (read 1321 more words →)

Nature < Nurture for AIs

This is a cross-link for https://scottviteri.github.io/post/nature-v-nurture-for-ais.

Let's imagine a hypothetical scenario where an AI is somehow trained in a way that is analogous to a human childhood in all of the relevant ways. Maybe it has a loving mother and family, maybe it has to learn to play well with peers, and maybe it has to learn to integrate into a larger social context. My expectation is that most members of the AI alignment community would expect this to create a fundamentally alien kind of intelligence. I will argue contrarily that nurture (the training data and training procedure) matters more than nature (transistors, gradient descent, and transformers) for a sufficiently capable AI.

This... (read 1898 more words →)

Conversationism

Research Direction: Be the AGI you want to see in the world

Intro

There seems to be a stark contrast between my alignment research on ontology maps and on cyborgism via Neuralink. I claim that the path has consisted of a series of forced moves, and that my approach to the second follows from my conclusions about the first. This post is an attempt to document those forced moves, in the hope that others do not have to duplicate my work.

Ontology maps are about finding shape semantic-preserving functions between the internal states of agents. Each commutative diagram denotes a particular way of training... (read 3430 more words →)

scottviteri, sudo, Lauro Langosco

Be the AGI you want to see in the world.

Epistemic status: highly speculative, authors are not neuroscientists.

Summary

It may be possible to enhance human intelligence via a brain-computer interface (BCI). We could put electrodes into a human brain, connect those electrodes to an artificial neural network, and train that network to predict and write neural activations.
This may present a technique for gradual uploading of human minds to computers that doesn’t require technology sufficient to create AGI in the first place.
The purpose of this article is to elicit feedback on this idea and, if it seems promising, encourage more work in this area.

Introduction: Goal and Idea

Goal

We suspect that it may be possible to develop... (read 2006 more words →)

REPL's and ELK

REPL's: a type signature for agents

In my previous $p o s t$ I talked about read-eval-print loops as providing a type signature for agents. I will now explain how you can quickly transition from this framework to an ELK solution. Notation is imported from that post.

Imagine we have two agents, a human and a strong AI, denoted H and M respectively. They both interact with the environment in lockstep, according to the following diagram.

We have the human's utility function $U_{H} : S_{H} \to Q$ , which is defined on the human's model of reality. We would like to lift $U_{H}$ to a version $U_{M} : S_{M} \to Q$ that the machine can use to influence the world in way that is agreeable to the human, which we can do by learning a mapping F : $S_{M} \to S_{H}$ and deriving $U_{M} = F \circ U_{H}$ .

But... (read 253 more words →)