MP — LessWrong

MP2dQuick Take

Someone, somewhere, should put some effort and answer the following question: is the cost of AI tasks increasing exponentially that at some point, LLMs coding at a certain time horizon will be more expensive than humans?

Replying toDistinguish between inference scaling and "larger tasks use more compute"

MP2d

Distinguish between inference scaling and "larger tasks use more compute"

I used to spend $2/mo in AI tokens for SOTA GPT-3.5 in mid 2022 for all my SOTA LLM needs. I now spend thousands, despite the insane price per token decline.

I have some moderate confidence the following is happening:

Tokens used to solve a task with greater difficulty is increasing faster than the amount of pricing decline per token.

Pricing decline of tokens is in the neighborhood of 90% per year. Token per task at SOTA is increasing 100x to 10,000x per year, in my experience.

MP4dQuick Take

Some take off scenario projects the compute available for LLM-led experiments?

I assume today you can model developments per human AI researcher as: number of experiments X number of FLOPS X research taste.

Over time, you have a lower bound on how many FLOPS you need for an experiment.

My question is how realistic is the improvement by having LLMs doing AI research, to the extent this means the LLMs will be conducting the experiments themselves. You'd probably need to 100-1000x the compute available to small scale experiments to see any meaningful acceleration.

Thoughts?

Inspired by this tweet: https://x.com/chrispainteryup/status/2020738025907712225?s=46

-1

MP10d

Meta has basically shut down FAIR after their Llama 4 fiasco, fired the lead and Yann, and they are starting again by creating a new lab called Meta Superintelligence Labs. The guys Alexander Wang has assembled STARTED working toghether like during the summer.

MP12dQuick Take

I plan to brief and lobby a presidential candidate with high single digits probability of getitng elected in my country about AI safety. What books or essays you guys think I should read as a last sprint ditch to get read to the task?

If the Superintelligence were near fallacy

13d

People will say:

"If the Superintelligence were near, OpenAI wouldn't be selling ads."
"If the Superintelligence were near, OpenAI wouldn't be adding adult content to ChatGPT."
"If the Superintelligence were near, OpenAI wouldn't be taking ecommerce referral fees."
"If the Superintelligence were near and about to automate software development, Anthropic wouldn't have a dozen of open roles for software developers."
"If the Superintelligence were near, OpenAI wouldn't be trying to take a cut of scientific innovations created with OpenAI models."
"If the Superintelligence were near, OpenAI employees wouldn't be selling OpenAI equity in the secondary market."
"If the Superintelligence were near, OpenAI wouldn't be doing acquisitions such as io, Roi, Torch, Sky, and Neptune."
"If the Superintelligence were near, OpenAI

... (read 2319 more words →)

MP17dQuick Take

New type of LLM psychosis: "LLM type 2 bipolatity"

In December I had very high levels of anxiety (started taking meds, difficulty sleeping) as part of the Claude Code moment. Sort of depression

Now I am having difficulty sleeping because I want to keep programming with ClawdBot, eating badly, and I unilateraly stopped taking the anxiety medication. Sort of mania.

Let's if this bipolarity disorder continues and see what my doctor thinks.

MP22d

Why you think Anthropic has low odds of succeeding?

Replying toClaude's new constitution

MP22d

Claude's new constitution

"We are updating Grok's constitution. We think it should be able to play many characters, like the waifu anime girl Ani, the truth-seeking AI assistant Grok, and the unhinged version MechaHitler."

What annoys me about alignment research is that it could be used for create evil models just by trivially plugging a minus sign in front of it.

-7

MP24dQuick Take

I have less confidence than most here in how much automated AI researchers will accelerate the takeoff. The difficulty of improving time horizons seem like a big data & compute problem. And it is also not clear how much you'll be able to iterate in other vectors.

But! I am relatively more confident that for a given level of intelligence, automated AI researchers will be brutal in optimizing the cost to run it. We should expect an acceleration of the price of intelligence. See NanoGPT and PostTrainBench.

This could also be a way the bubble pops. At some point, running more expensive models stop making economic sense for a given task.

Imagine if by Summer 2028 we can run a Claude Opus 6-level model in your own Mac. Will Claude Opus 7 be that much better if you're not doing advanced research? What about Summer 2029?

No wonder we are hearing about the possibility of a 2026 Anthropic IPO.

-1

MP24d

Of course being right is better than being wrong. Ideally he should know the exact date of the arrival of the Superintelligence and organize finances for that.

But it seems to me that he has the best shot of creating the AI god with his current process.

-1

Many many things in the world are explained by society having an unconscious visceral forecast of AGI timelines in the late 2020s. Many actors behaviors make more sense if you assume they have late 2020s timelines.

Why the end of neoliberalism and the post-war world? Late 2020s timelines. They understand having raw power will be relatively more important than having non-zero-sum relationship with your allies and trading partners.
Why the 2nd Cold War? Late 2020s timelines. U.S. and China know that being ahead during the late 2020s probably means being ahead forever.
Why did Russia invade Ukraine? Late 2020s timelines. Russia knows they couldn't do it later.
Why Gen Z isn't obsessed with work but they

... (read more)

Someone posted on X's fintwit:

Claude Code virality is first big AI hit which inspires lots of short ideas but almost no longs. Who benefits besides anthropic or ai infra trade that’s already max long?

To me, this is more evidence of dystopia. Maybe I am more optimistic than the average LessWronger who believes in apocalypse, but this updates me towards a very weird future.

This hints to a future where AI is extremely deflationary.

It really helps to think about the world as if people were rational working under full information.

It really makes sense for China to invade Taiwan as soon as this summer.

1- China won't be able to win the long timelines world many policymakers are planning for. The David Sacks/Jensen Hwang argument is that by precluding China from the best AI accelerators, they'll eventually have their own domestic supply the U.S. can't control. When that happens, the U.S. will have a robot army they can cheaply deploy to defend the Taiwan strait.

2- Taiwan is really central for U.S. AI development. Really. Not only TSMC, but Micron, Mediatek, Alchip, all the ODMs like Foxconn.

3- China... (read more)

•••

It might be the case that 2025 was the culmination of many low-hanging fruits driving AI development than an actual take-off.

MP's Shortform

1mo

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

ASI is often assumed as the all knowing first-principles thinker.

Sam Altman once said: "We don't know how we'll make money. Maybe we'll create the AGI and then ask it to make us one billion dollars". Despite that, my read of the Bitter Lesson is that empiricism, not cleverness, is what have been pushing the AI endeavor forward. I have seen my fair share of researchers during the 2010s creating ontologies and programming in Lisp, only to get obliterated by deep learning and brute force.

Machine learning has obliterated humans in fields such as chess, computer vision, and is on track to do so in math and programming. These are themes where if you... (read more)

Can singularity emerge from transformers?

One of the most aesthetically pleasing facts in computer science is that after compiling by hand one compiler, you can you your compiled compiler to compile an ever-larger compiler.

One historical fact that most people seem to forget about artificial intelligence is that the first attempt people had, more than half-century ago, was lisp machines. A machine that writes code. As recently as 2018, I was forced to do my course on Data Structures and Algorithms using Lisp because one of my professors is still a believer that these statistical approaches to NLP aren't good enough.

I don't understand exactly how you can approach singularity in the transformer paradigm. It seems to me that... (read more)

In the age of modern AI (LLMs and beyond), is data still the new oil?

In the age of big data and machine learning, it was common to say that data was the new oil. Every big company spent a lot of money creating their Hadoop instances, hiring data scientists, to discover new things from the data they had. It's easy to see how the big data promise failed to live up to its promise.

Today, when companies go train their LLMs, they can use databases that are publicly available: the Common Crawl was 82% of the raw tokens used to train GPT-3 and the latest version is 428TB. The novel word in town is "high value tokens", in this world, adding some data generated by humans exclusively... (read 564 more words →)

Is AGI communist?

I think a lot of people dislike central planning states not because they dislike efficiency, welfare, or lack of liberties, but because they distrust anyone central party to make decisions on behalf of society.

Well, if you have greater than life intelligence like AGI, you could do central planning. At least, you could think you can do central planning.

Is it possible that AGIs will have left-leaning ideas, and ultimately be communists, because they see themselves as smart and that things would be better off if humans let them to allocate resources instead of markets.

-10

LESSWRONG
LW

LESSWRONG
LW

MP

If the Superintelligence were near fallacy

In the age of modern AI (LLMs and beyond), is data still the new oil?

Can singularity emerge from transformers?

MP's Shortform

MP

MP

If the Superintelligence were near fallacy

MP's Shortform

Can singularity emerge from transformers?

In the age of modern AI (LLMs and beyond), is data still the new oil?

Is AGI communist?

MP

If the Superintelligence were near fallacy

In the age of modern AI (LLMs and beyond), is data still the new oil?

Can singularity emerge from transformers?

MP's Shortform

MP

MP

If the Superintelligence were near fallacy

MP's Shortform

Can singularity emerge from transformers?

In the age of modern AI (LLMs and beyond), is data still the new oil?

Is AGI communist?