I think most of the imitation happens in the pretraining. I don't know about o1, but DeepSeek v3 is at minimum trained on a ton of 4o outputs, although they are slightly cagey about this. Just the first thing I tried, I had ChatGPT write a sock poem:

4o's poem:

Socks, the keepers of toes so warm,
Guardians of feet through life's storm.
In stripes, in polka dots, or plain,
They dance with us through joy and pain.

A cozy pair on winter’s chill,
Or mismatched mates—yet charming still.
From drawer to floor, they roam and stray,
But where one goes, the other may.

Oh, humble socks, so oft unseen,
Yet vital threads of life's routine.
A quiet comfort, soft and neat,
The unsung heroes of weary feet.
 

Then I gave v3 just the first two stanzas, and asked it to continue the poem:

v3's continuation:

They hide in shoes, they lounge in beds,
Sometimes they’re lost, or so it’s said.
Yet always there, a faithful friend,
From morning’s start to evening’s end.

They cushion steps, they soften blows,
Through every stride, their comfort shows.
Oh, humble socks, so oft unseen,
The quiet heroes of routine.

So here’s to socks, both bold and meek,
For every day, for every week.
A simple joy, a small delight,
They keep us snug, from day to night.

 

The shot in the dark guess of the "humble socks, so oft unseen... routine" couplet is a fucking soul read. v3 knows 4o, in a way that I kind of hope no model ever knows a person.

As a quirk of the US patent system, patenting an idea doesn't actually require implementing it or proving that the implementation works. As a result, if you want to try this, you should patent ideas that seem good or bad to you in a scattershot manner, but should not do the actual underlying capabilities research. Then, you get to sue if someone else independently comes up with the patented idea and actually finds success with it, but you don't contribute to actual capabilities advances any more than an ideas guy posting to /r/singularity.

(Source: I have actually patented a thing, but do not really endorse this decision and haven't sued anyone about it.  I am not a patent lawyer)

Lets imagine a 250 IQ unaligned paperclip maximizer that finds itself in the middle of an intelligence explosion. Let’s say that it can’t see how to solve alignment. It needs a 350 IQ ally to preserve any paperclips in the multipolar free-for-all. Will it try building an unaligned utility maximizer with a completely different architecture and 350 IQ?

I’d imagine that it would work pretty hard to not try that strategy, and to make sure that none of its sisters or rivals try that strategy. If we can work out what a hypergenius would do in our shoes, it might behoove us to copy it, even if it seems hard.

So if alignment is as hard as it looks, desperately scrabbling to prevent recursive superintelligence should be an extremely attractive instrumental subgoal. Do we just lean into that?

I tried to learn to write before I had important things to say and it basically didn’t work. I had to go through the work of coming up with genuinely valuable ideas and then wreck the presentation of those ideas via bad writing. My more recent publications, I’m actually very happy with the writing.

The first couple times a surgeon does an operation, patient outcomes suck. Alas, there’s no other way to make experienced surgeons. My guess is that writing is similar, and I’m very glad that important experiences and ideas are way less valuable than patients: I would emotionally struggle with becoming a surgeon.

Hi! I've had some luck making architectures equivariant to a wider zoo of groups: my most interesting published results are getting a neural network to output a function, and invert that function if the inputs are swapped (equivariant to group of order 2, https://arxiv.org/pdf/2305.00087) and getting a neural network with two inputs to be doubly equivariant to translations: https://arxiv.org/pdf/2405.16738

These are architectural equivariances, and as expected that means they hold out of distribution.

If you need an architecture equivariant to a specific group, I can probably produce that architecture; I've got quite the unpublished toolbox building up. In particular, explicit mesa-optimizers are actually easier to make equivariant- if each mesa-optimization step is equivariant to a small group, then the optimization process is tyically equivariant to a larger group

There’s an easy way to turn any mathematical answer-based benchmark into a proof-based benchmark and it doesn’t require coq or lean or any human formalization of the benchmark design: just let the model choose whether or not to submit an answer for each question, and score the model zero for the whole benchmark if it submits any wrong answers.

is this Leverage adjacent?

It’s not AGI, but for human labor to retain any long-term value, there has to be an impenetrable wall that AI research hits, and this result rules out a small but nonzero number of locations that wall might’ve been.

“Scaling is over” was sort of the last hope I had for avoiding the “no one is employable, everyone starves” apocalypse. From that frame, the announcement video from openai is offputtingly cheerful.