aribrill — LessWrong

LESSWRONG
LW

Progress update: synthetic models of natural data

1mo

This post presents a brief progress update on the research I am doing as a part of the renormalization research group at Principles of Intelligence (PIBBSS). The code to generate synthetic datasets based on the percolation data model is available in this repository. It employs a newly developed algorithm to construct a dataset in a way that explicitly and iteratively reveals its innate hierarchical structure. Increasing the number of data points corresponds to representing the same dataset at a more fine-grained level of abstraction.

Introduction

Ambitious mechanistic interpretability requires understanding the structure that neural networks uncover from data. A quantitative theoretical model of natural data's organizing structure would be of great value for AI... (read 888 more words →)

Investigating Neural Scaling Laws Emerging from Deep Data Structure

Nathaniel Mitrani

Nathaniel Mitrani, aribrill

4mo

This is a summary of the project I undertook during SPAR, mentored by Ari Brill. I have recently had to prioritize other work, and found more questions than answers, as is often the case. I think a write-up will be useful both for coming back to this and interesting for the findings (and non-findings!) it presents.

Introduction

Real-world data is implicitly structured by the reality it represents, not just its surface-level format. AI systems can learn surface patterns, deeper structures, or a combination of both. Let's illustrate this with an example.

Take (the concept of) a dog, for instance. This dog can be represented by a picture, a GIF, a paragraph that describes it, a... (read 2188 more words →)

AI Safety x Physics Grand Challenge

Lauren Greenspan

Lauren Greenspan, aribrill

7mo

Join us for the AI Safety x Physics Grand Challenge, a research hackathon designed to engage physicists in technical AI safety research. While we expect LessWrong community members with both technical AI safety and physics expertise to benefit most from this event, we encourage anyone interested in exploring this intersection to sign up.

Dates: July 25th to July 27th (this weekend)

Location: Remote, with in-person hubs in several locations

Prizes: $2,000 total prize money

Apart Research is running the hackathon, in collaboration with PIBBSS and Timaeus. Hackathon speakers include Jesse Hoogland (Timaeus), Paul Riechers (Simplex), and Dmitry Vaintrob (PIBBSS). Participants will get research support from mentors with expertise spanning the physics and AI safety space, including Martin... (read 2135 more words →)

aribrill8mo

In your usage, are "scheming" and "deceptive alignment" synonyms, or would you distinguish those terms in any way?

Replying toRenormalization Roadmap

aribrill10mo

Renormalization Roadmap

This post is great to see, I think renormalization is a very exciting direction for AI safety research!

First, as one possible way to represent the real-world, we can think of representation_0 as a low-energy description of the dataset: IR_0.
If the NN is capable of learning a meaningful generalization of the data, representation_0 flows to representation_1 (now UV_1) via an implicit RG flow to higher energies. Instead of throwing information away, flowing to UV_1 adds structure that allows it to more reliably adapt to unseen information.

Shouldn't this go the other way, with representation_0 being UV and representation_1 being IR? A NN compresses the input representation (data) to obtain a coarse-grained output representation (label). The ability to throw away information, i.e. the irrelevant noise w.r.t. the target function, is what enables generalization to unseen inputs differing in fine-grained details.

Neural Scaling Laws Rooted in the Data Distribution

aribrill

This is a linkpost for my recent research paper, which presents a theoretical model of power-law neural scaling laws.

Abstract:

Deep neural networks exhibit empirical neural scaling laws, with error decreasing as a power law with increasing model or data size, across a wide variety of architectures, tasks, and datasets. This universality suggests that scaling laws may result from general properties of natural learning tasks. We develop a mathematical model intended to describe natural datasets using percolation theory. Two distinct criticality regimes emerge, each yielding optimal power-law neural scaling laws. These regimes, corresponding to power-law-distributed discrete subtasks and a dominant data manifold, can be associated with previously proposed theories of neural scaling, thereby grounding

... (read 155 more words →)

Replying toDmitry's Koan

aribrill1y

Dmitry's Koan

The notion of a precision scale for interpretability is really interesting, particularly the connection with generalization/memorization. This seems like a fruitful concept to develop further.

How, then can we operationalize the loss scale of a phenomenon? Well, one way to do this is to imagine that we have some "natural" complexity parameter c that can be varied (this can be a parameter tuning model size, training length, etc.).

It could be interesting to think about the interpretation of different possible complexity parameters here. You might expect these to give rise to distinct but related notions of generalization. Here, I’m drawing on intuitions from my work connecting scaling laws to data distributions, though I hadn’t... (read 395 more words →)

aribrill1y

Thanks for the great writeup.

Superposition ("local codes") require sparsity, i.e. that only few features are active at a time.

Typo: I think you meant to write distributed, not local, codes. A local code is the opposite of superposition.

Computational Complexity as an Intuition Pump for LLM Generality

aribrill

With sufficient scale and scaffolding, LLMs will improve without bound on all tasks to become superhuman AGI, to the extent they haven’t already. No, wait! LLMs are dead-end pattern-matching machines fundamentally incapable of general reasoning and novel problem solving. Which is it?

I’ll call these opposing points of view “LLM Scaler” and “LLM Skeptic”. The first appears to be held by the big AGI labs and was recently exemplified by Leopold Aschenbrenner’s Situational Awareness series, while the second is influenced by cognitive science and can be associated with researchers such as François Chollet, Gary Marcus, and Melanie Mitchell. This post loosely generalizes these two stances, so any misrepresentation or conflation of individuals’ viewpoints is... (read 687 more words →)

Replying toNatural abstractions are observer-dependent: a conversation with John Wentworth

aribrill2y

Natural abstractions are observer-dependent: a conversation with John Wentworth

Short answer: some goals incentivize general intelligence, which incentivizes tracking lots of abstractions and also includes the ability to pick up and use basically-any natural abstractions in the environment at run-time.
Longer answer: one qualitative idea from the Gooder Regulator Theorem is that, for some goals in some environments, the agent won't find out until later what its proximate goals are. As a somewhat-toy example: imagine playing a board game or video game in which you don't find out the win conditions until relatively late into the game. There's still a lot of useful stuff to do earlier on - instrumental convergence means that e.g. accumulating resources and gathering information and building general-purpose

... (read 665 more words →)

Replying toMeetup : Yale: Initial Meetup

aribrill12y

Meetup : Yale: Initial Meetup

Unfortunately I am busy from 2-5 on Sundays, but I would certainly like to attend a future Yale meetup at some other time.

Replying toRationality Quotes August 2013

aribrill13y

Rationality Quotes August 2013

In 2002, Wizards of the Coast put out Star Wars: The Trading Card Game designed by Richard Garfield.

As Richard modeled the game after a miniatures game, it made use of many six-sided dice. In combat, cards' damage was designated by how many six-sided dice they rolled. Wizards chose to stop producing the game due to poor sales. One of the contributing factors given through market research was that gamers seem to dislike six-sided dice in their trading card game.

Here's the kicker. When you dug deeper into the comments they equated dice with "lack of skill." But the game rolled huge amounts of dice. That greatly increased the consistency. (What I mean by this is that if you rolled a million dice, your chance of averaging 3.5 is much higher than if you rolled ten.) Players, though, equated lots of dice rolling with the game being "more random" even though that contradicts the actual math.

Mark Rosewater, Kind Acts of Randomness

Replying toRationality Quotes June 2013

aribrill13y

Rationality Quotes June 2013

Why is there that knee-jerk rejection of any effort to "overthink" pop culture? Why would you ever be afraid that looking too hard at something will ruin it? If the government built a huge, mysterious device in the middle of your town and immediately surrounded it with a fence that said, "NOTHING TO SEE HERE!" I'm pretty damned sure you wouldn't rest until you knew what the hell that was -- the fact that they don't want you to know means it can't be good.

Well, when any idea in your brain defends itself with "Just relax! Don't look too close!" you should immediately be just as suspicious. It usually means something ugly is hiding there.

David Wong, The 5 Ugly Lessons Hiding in Every Superhero Movie

Replying toRationality Quotes January 2013

aribrill13y

Rationality Quotes January 2013

"How is it possible! How is it possible to produce such a thing!" he repeated, increasing the pressure on my skull, until it grew painful, but I didn't dare object. "These knobs, holes...cauliflowers -" with an iron finger he poked my nose and ears - "and this is supposed to be an intelligent creature? For shame! For shame, I say!! What use is a Nature that after four billion years comes up with THIS?!"

Here he gave my head a shove, so that it wobbled and I saw stars.

"Give me one, just one billion years, and you'll see what I create!"

Stanislaw Lem, "The Sanatorium of Dr. Vliperdius" (trans. Michael Kandel)

Replying toIntelligence explosion in organizations, or why I'm not worried about the singularity

aribrill13y

Intelligence explosion in organizations, or why I'm not worried about the singularity

That's certainly true. It seems to me that in this case, sbenthall was describing entities more akin to Google than to the Yankees or to the Townsville High School glee club; "corporations" is over-narrow but accurate, while "organizations" is over-broad and imprecise.