One winter a grasshopper, starving and frail, approaches a colony of ants drying out their grain in the sun to ask for food, having spent the summer singing and dancing.
Then, various things happen.
I appeared on the 80,000 Hours podcast. I discussed a bunch of points on misalignment risk and AI control that I don't think I've heard discussed publicly before.
Transcript + links + summary here; it's also available as a podcast in many places.
1I occasionally get texts from journalists asking to interview me about things around the aspiring rationalist scene. A few notes on my thinking and protocols for this:
- I generally think it is pro-social to share information with serious journalists on topics of clear public interest.
- By-default I speak with them only if their work seems relatively high-integrity. I like journalists whose writing is (a) factually accurate, (b) boring, and (c) do not feel to me to have an undercurrent of hatred for their subjects.
- By default I speak with them off-the-record, and then offer to send them write-ups of the things I said that they want to quote. This has gone quite well. I've felt comfortable speaking in my usual fashion without worrying about nailing each and every phrasing. Then I ask what they're interested in quoting, and I send them (typically a 1-2 page) google doc on those topics (largely re-stating what I already said to them, and making some improvements / additions). Then they tell me which quotes they want to use (typically cutting many sentences or paragraphs half-way). Then I make one or two slight edits and give them explicit permission to quote. I think this has gone quite well and they've felt my quotes were substantive and improvements.
- For the New York Times, I am currently trying out the policy of "I am happy to chat off-the-record. I will also offer quotes by my usual protocol, but I will only give them conditional on you including a mention that I disapprove of the NYT's de-anonymization policies (which I bring up due to your reckless and negligent behavior that upturned the life of a beloved member of my community)." I am about to try this for the first time, and I expect they will thus not want to use my quotes, and that's fine by me.
2
1
1Popular Comments
Recent Discussion
Meta released 3 models from the Llama 4 family herd today.
- The 109B (small) and 400B (mid) models to be MoE models with 17B active param each. The large model is 2T MoE model iwth 288B active param instead.
- The small model is called Scout. The middle model is called Maverick. The big one is called Behemoth.
- Natively multimodal and much longer context length. 10M (!!) for the small one and 1M for the mid sized one.
- Maverick is currently number 2 on LLM arena. (Number 5 if style controlled)
- We don't have the 2T big model available yet. It is used as a distillation model for Maverick and Scout.
- Behemoth beats Claude 3.7 (not thinking), GPT 4,5 and Gemini 2.0 Pro in various benchmarks (LiveCodeBench, Math-500, GPQA Diamond, etc)
- Scout is meant to work on a single H100.
https://ai.meta.com/blog/llama-4-multimodal-intelligence/
This year's Spring ACX Meetup everywhere in SAINT PAUL.
Location: Davanni's Pizza ; 41 Cleveland Ave S, St Paul, MN 55105 – https://plus.codes/86P8WRQ6+XX
Please RSVP on lesswrong so i know how much food to get. Also note that regrettably if you're a vegan davanni's has salad and little else (though i will be getting the salad).
Contact: ironlordbyron@gmail.com
“In the loveliest town of all, where the houses were white and high and the elms trees were green and higher than the houses, where the front yards were wide and pleasant and the back yards were bushy and worth finding out about, where the streets sloped down to the stream and the stream flowed quietly under the bridge, where the lawns ended in orchards and the orchards ended in fields and the fields ended in pastures and the pastures climbed the hill and disappeared over the top toward the wonderful wide sky, in this loveliest of all towns Stuart stopped to get a drink of sarsaparilla.”
— 107-word sentence from Stuart Little (1945)
Sentence lengths have declined. The average sentence length was 49 for Chaucer (died 1400), 50...
Semicolons are unnecessary? That doesn’t go far enough. Cormac McCarthy got rid of quotation marks, most commas, and almost exterminated the colon.
I frequently hear people make the claim that progress in theoretically physics is stalled, partly because all the focus is on String theory and String theory doesn't seem to pan out into real advances.
Believing it fits my existing biases, but I notice that I lack the physics understanding to really know whether or not there's progress. What do you think?
That's an inference, presumably Adam believes that for object-level reasons, which could be supported by eg looking at the age at which physicists make major advancements[1] and the size of those advancements.
Edit: But also this wouldn't show whether or not theoretical physics is actually in a rut, to someone who doesn't know what the field looks like now.
Adjusted for similar but known to be fast moving fields like AI or biology to normalize for facts like eg the academic job market just being worse now than previously. ↩︎
What is solitude? If you are alone in a forest reading a book, miles away from any other humans, is that solitude?
No, it is not. Cal Newport's book Digital Minimalism is where I learned this.
The key to solitude is being isolated from the input of other minds. When you're alone in the forest, there aren't any people physically in your vicinity, but the author of the book you're reading is very much influencing your thoughts.
Think about it: imagine if the author was sitting next to you in the forest on a bench telling you about their thoughts on, I don't know, congestion pricing. How different is that from them writing a book about congestion pricing and you reading it? In both scenarios you are basically consuming the...
I am a policy researcher and data protection officer working in AI Governance for an European corporation. I do not have a ML background, and I am a lawyer... which is precisely why I am here, seeking your expertise.
Over the past few months, I’ve been working at the intersection of AI governance and technical alignment research, trying to understand how AI safety concepts can be translated into regulatory structures that preserve both nuance and enforceability.
In doing so, I’ve noticed recurring confusion around terminology, especially between what we mean by “AI Safety” vs. “AI Security”.
Mentioning these disciplines can carry different connotations depending on whether you're an ML engineer, a policy advisor, or a safety researcher focused on x-risk. But, as I read papers that aim to inform policy,...
I will bite.
First of all, I appreciate the effort of trying to communicate better and hammering down the neat borders of word and how they used across domains - especially for words that are often used interchangeably and carelessly.
TLDR: Sometimes posts just get unlucky! And your style is on the verbose side and I am still somewhat confused about your value props.
It seems like your frustration is from a lack of responses - often, a lack of response might just be luck based and how the LW algorithm works (exponential time decay). Maybe yo...
Just a quick thought. It’s intriguing how often we frame the future in terms of present and past motivators—trade, money, ownership, equality, you name it. All of it’s fair game, considering we’re still human, at least for now.
If we try to sketch out possible futures through a cone of probabilities, especially assuming we reach AGI or even a more potent ASI, we tend to land on a handful of familiar scenarios and their close cousins. But there’s something I think we overlook—something that, to me, feels more likely than the usual suspects we keep circling. Here’s my stab at summing it up.
Often discussed
- AI Takeover - Terminator meets Matrix
- AI Takeover - Buddy Pixar Movie
- AI + Autocracy: Tremendously greater inequality through control; need for complaince.
- AI +
Thanks for sharing this and for the examples layed out. I was not familiar with all of them, though many. but I did omit stating that I meant outside of fiction. My assumption is still relatively short timeframes of 5 to 15 years. Under those assumptions I dont necessarily see scenario 1 or 7 being more likely than scenario 8.
Quick note. I see a show like Upload being a potential representation of a facet of these scenarios. For example scenarios 2 to 7 could all have widespread virtual realities for the common person or those who opt out willingly o...
My thoughts on the recently posted story.
Caveats
- I think it's great that the AI Futures Project wrote up a detailed scenario.
- I enjoy it.
- Every part of the story i didn't comment on here is either fine or excellent.
- This is one of the most realistic scenarios i've read.
- All detailed predictions contain errors.
- The authors of this scenario don't claim it's the most likely future.
- If the speed of 2018-2025 was the typical rate of progress in software, then AI 2027 would be realistic.
Core Disagreements
- Early 2026: OpenBrain is making algorithmic progress 50% faster.
- As with many parts of this scenario, i think this is plausible but too fast. 150% productivity is a lot in business terms, & the scenario doesn't provide much detail for why this is 150% as opposed to 110%. In my
I don't really understand how a local copy of the weights gives the terrorists more practical control over the software's alignment. I don't think it's easy to manually tweak weights for so specific a purpose. Maybe they just mean the API is doing a good job of blocking sketchy requests?
You can finetune models for any specific purpose: just provide a few datapoints and train. The more specific the purpose, the easier tweaking the weights is, not harder. (Surely, if nothing else, you've seen all of the LoRAs and other things for finetuning image gener...
