I don't understand why people rave so much about Claude Code etc., nor how they really use these agents. The problem is not capability--sure, today agents can go far without stumbling or losing the plot. The problem is that they will go not in the direction I want.
It's because my product vision, architectural vision, and code quality "functions" are complex: very tedious to express in CLAUDE/AGENTS .md, and often hardly expressible in language at all. "I know it when I see it." Hence keeping agent "on a short leash" (Karpathy)--in Cursor.
This makes me thin...
Has Tyler Cowen ever explicitly admitted to being wrong about anything?
Not 'revised estimates' or 'updated predictions' but 'I was wrong'.
Every time I see him talk about learning something new, he always seems to be talking about how this vindicates what he said/thought before.
Gemini 2.5 pro didn't seem to find anything, when I did a max reasoning budget search with url search on in aistudio.
5
2
2
1this is evidence that tyler cowen has never been wrong about anything
2Every now and then in discussions of animal welfare, I see the idea that the "amount" of their subjective experience should be weighted by something like their total amount of neurons. Is there a writeup somewhere of what the reasoning behind that intuition is? Because it doesn't seem intuitive to me at all.
From something like a functionalist perspective, where pleasure and pain exist because they have particular functions in the brain, I would not expect pleasure and pain to become more intense merely because the brain happens to have more neurons. Rather...
I don't have a detailed writeup, but this seems straightforward enough to fit in this comment: you're conducting your moral reasoning backwards, which is why it looks like other people have a sophisticated intuition about neurobiology you don't.
The "moral intuition"[1] you start with is that insects[2] aren't worth as much as people, and then if you feel like you need to justify that, you can use your knowledge of the current best understanding of animal cognition to construct a metric that fits of as much complexity as you like.
I'd call m
Gary Marcus asked me to make a critique of his 2024 predictions, for which he claimed that he got "7/7 correct". I don't really know why I did this, but here is my critique:
For convenience, here are the predictions:
I think the best way to evaluate them is to invert every one of them, and then see whether the version you wrote, or the i...
9
1
1One lesson you should maybe take away is that if you want your predictions to be robust to different interpretations (including interpretations that you think are uncharitable), it could be worthwhile to try to make them more precise (in the case of a tweet, this could be in a linked blog post which explains in more detail). E.g., in the case of "No massive advance (no GPT-5, or disappointing GPT-5)" you could have said "Within 2024 no AI system will be publicly released which is as much of a qualitative advance over GPT-4 in broad capabilites as GPT-4 is ...
i made a thing!
it is a chatbot with 200k tokens of context about AI safety. it is surprisingly good- better than you expect current LLMs to be- at answering questions and counterarguments about AI safety. A third of its dialogues contain genuinely great and valid arguments.
You can try the chatbot at https://whycare.aisgf.us (ignore the interface; it hasn't been optimized yet). Please ask it some hard questions! Especially if you're not convinced of AI x-risk yourself, or can repeat the kinds of questions others ask you.
Send feedback to ms@contact.ms.
A coup...
Nope, I’m somewhat concerned about unethical uses (eg talking to a lot of people without disclosing it’s ai), so won’t publicly share the context.
If the chatbot answers questions well enough, we could in principle embed it into whatever you want if that seems useful. Currently have a couple of requests like that. DM me somewhere?
Stampy uses RAG & is worse.
I was a relatively late adopter of the smartphone. I was still using a flip phone until around 2015 or 2016 ish. From 2013 to early 2015, I worked as a data scientist at a startup whose product was a mobile social media app; my determination to avoid smartphones became somewhat of a joke there.
Even back then, developers talked about UI design for smartphones in terms of attention. Like, the core "advantages" of the smartphone were the "ability to present timely information" (i.e. interrupt/distract you) and always being on hand. Also it was small, so anyth...
7
3
1My main concern with heavy LLM usage is what Paul Graham discusses in Writes and Write-Nots. His argument is basically that writing is thinking and that if you use LLM's to do your writing for you, well, your ability to think will erode.
random brainstorming ideas for things the ideal sane discourse encouraging social media platform would have:
2111have some threaded chat component bolted on (I have takes on best threading system).
I wish to hear these takes.
The "uncensored" Perplexity-R1-1776 becomes censored again after quantizing
Perplexity-R1-1776 is an "uncensored" fine-tune of R1, in the sense that Perplexity trained it not to refuse discussion of topics that are politically sensitive in China. However, Rager et al. (2025)[1] documents (see section 4.4) that after quantizing, Perplexity-R1-1776 again censors its responses:
I found this pretty surprising. I think a reasonable guess for what's going on here is that Perplexity-R1-1776 was finetuned in bf16, but the mechanism that it learned for non-refus...
5not enough noise in fine-tuning training then
I'd categorize that as an exfohazard rather than an infohazard.
Info on how to build a nuke using nothing but parts of a microwave doesn't harm the bearer, except possibly by way of some other cognitive flaw/vulnerability (e.g. difficulty keeping secrets)
Maybe "cognitohazard" is a closer word to the thing I'm trying to point towards. Though, I would be interested in learning about pure infohazards that aren't cognitohazards.
(If you know of one and want to share it with me, it may be prudent to dm rather than comment here)
So, first: The logistical details of reducing wild impact biomass are mooted by the fact that I meant it as a reductio, not a proposal. I have no strong reason to think that spraying insecticide would be a better strategy than gene drives or sterile insect technique or deforestation, or that DDT is the most effective insecticide.
To put rough numbers on it: honeybees are about 4e-7 by count or 7e-4 by biomass of all insects (estimate by o3). There is no such extreme skew for mammals and birds (o3). While domesticated honeybees have some bad things happen to...
There was a recent post titled "Spaced Repetition Systems Have Gotten Way Better": https://domenic.me/fsrs/
It mentions this:
But what’s less widely known is that a quiet revolution has greatly improved spaced repetition systems over the last couple of years, making them significantly more efficient and less frustrating to use. The magic ingredient is a new scheduling algorithm known as FSRS, by Jarrett Ye.
I was skeptical, but I tried getting into spaced repetition again and I can say that the FSRS algorithm feels just magical. I often find that I'm just bar...
So, I was wondering whether this is usable in anki, and indeed, there appears to be a simple setting for it without even having to install a plugin, as described here in 4 easy steps. I'll see if it makes a notable difference.
Not so relatedly, this made me realize a connection I hadn't really thought about before: I wish music apps like Spotify would use something vaguely like spaced repetition for Shuffle mode. In the sense of finding some good algorithm to predict, based on past listening behavior, which song in a playlist the user is most likely to curr...
In a thread which claimed that Nate Soares radicalized a co-founder of e-acc, Nate deleted my comment – presumably to hide negative information and anecdotes about how he treats people. He also blocked me from commenting on his posts.
The post concerned (among other topics) how to effectively communicate about AI safety, and positive anecdotes about Nate's recent approach. (Additionally, he mentions "I’m regularly told that I’m just an idealistic rationalist who’s enamored by the virtue of truth" -- a love which apparent...
1That said, my feeling is Trump et al. weren't reacting against any specific woke activism, but very woke policies (and opinions) which resulted from the activism.
I don't think this is true, and that indeed the counter-reaction is strongly to the woke activism. My sense is a lot of current US politics stuff is very identity focused, the policies on both sides matter surprisingly little (instead a lot of what is going on is something more like personal persecution of the outgroup and trying to find ways to hurt them, and to prop up your own status, which actually ends up with surprisingly similar policies on both ends).
Polymarket is not liquid enough to justify betting full-time.
Optimistically I expect if I invested $5k and 4 days per month for 6 months, I could make $7k +- $2k expected returns. Or $0-4k profit. I would have to split up the $5k into 6-7 bets and monitor them all separately.
I could probably make similar just working at a tech job.
I’ve been trying to understand modules for a long time. They’re a particular algebraic structure in commutative algebra which seems to show up everywhere any time you get anywhere close to talking about rings - and I could never figure out why. Any time I have some simple question about algebraic geometry, for instance, it almost invariably terminates in some completely obtuse property of some module. This confused me. It was never particularly clear to me from their definition why modules should be so central, or so “deep.”
I’m going to try to explain the ...
Historically commutative algebra came out of algebraic number theory, and the rings involved - Z,Z_p, number rings, p-adic local rings... - are all (in the modern terminology) Dedekind domains.
Dedekind domains are not always principal, and this was the reason why mathematicians started studying ideals in the first place. However, the structure of finitely generated modules over Dedekind domains is still essentially determined by ideals (or rather fractional ideals), reflecting to some degree the fact that their geometry is simple (1-dim regular Noetherian domains).
This could explain why there was a period where ring theory developed around ideals but the need for modules was not yet clarified?
Here are a cluster of things. Does this cluster have a well-known name?
Is the thing you're trying to label the peculiar confirmation bias where people instead of interpreting evidence to confirm to what they prefer or would like to be true, only to what they believe to be true - even if from their perspective it is pessimistic?
Or are you looking for a label for "this is unpopular therefore it can't win" as a specific kind of self-fulfilling prophecy? Like an inverted Keynesian beauty contest?
i learned something about agency when, on my second date with my now-girlfriend, i mentioned feeling cold and she about-faced into the nearest hotel, said she left a scarf in a room last week, and handed me the nicest one out of the hotel’s lost & found drawer
— @_brentbaum, tweet (2025-05-15)
you can just do things?
— @meansinfinity
not to burst your bubble but isn't this kinda stealing?
— @QiaochuYuan
What do people mean when they say "agency" and "you can just do things"? I get a sense it's two things, and the terms "agency" and "you can just d...
Here's a riddle: A woman falls in love with a man at her mother's funeral, but forgets to get contact info from him and can't get it from any of her acquaintances. How could she find him again? The answer is to kill her father in hopes that the man would come to the funeral.
It reminds me of [security mindset](https://www.schneier.com/blog/archives/2008/03/the_security_mi_1.html), in which thinking like an attacker exposes leaky abstractions and unfounded assumptions, something that is also characteristic of being agentic and "just doing things."
TAP for fighting LLM-induced brain atrophy:
"send LLM query" ---> "open up a thinking doc and think on purpose."
What a thinking doc looks varies by person. Also, if you are sufficiently good at thinking, just "think on purpose" is maybe fine, but, I recommend having a clear sense of what it means to think on purpose and whether you are actually doing it.
I think having a doc is useful because it's easier to establish a context switch that is supportive of thinking.
For me, "think on purpose" means:
I'm often in situations where either
a) I do basically expect the LLMs to get the right answer, and for it to be easily checkable. (like, I do in fact have a lot of boilerplate code to write)
and/or b) my current task is sufficiently tree structured, that it's pretty cheap to spin up an LLM to tackle one random subproblem while I mostly focus on a different thing. And the speedup from this is pretty noticeable. Sometimes the subproblem is something I expect it to get right, sometimes I don't really expect it to, BUT, there's a chance it will, and meanwhile I...
The motte and bailey of transhumanism
Most people on LW, and even most people in the US, are in favor of disease eradication, radical life extension, reduction of pain and suffering. A significant proportion (although likely a minority) are in favor of embryo selection or gene editing to increase intelligence and other desirable traits. I am also in favor of all these things. However, endorsing this form of generally popular transhumanism does not imply that one should endorse humanity’s succession by non-biological entities. Human “uploads” are much ...
computers are just not better at biology than biology. anything you'd do with a computer, once you're advanced enough to know how, you'd rather do by improving biology
I share a similar intuition but I haven't thought about this enough and would be interested in pushback!
it's not transhumanism, to my mind, unless it's to an already living person. gene editing isn't transhumanism
You can do gene editing on adults (example). Also in some sense an embryo is a living person.
The Sasha Rush/Jonathan Frankle wager: https://www.isattentionallyouneed.com/ is extremely unlikely to be untrue by 2027, but it's not because another architecture might not be better; it's because it asks whether a transformer-like model will be sota . I think it is more likely that transformers are a proper subset of a class of generalized token/sequence mixers. Even SSMs when unrolled into a cumulative sum are a special case of linear attention.
Personally I do believe that there will be a deeply recurrent method that is transformer-like to succeed the transformer architecture, even though this is an unpopular opinion.
I changed my mind on this after seeing the recent literature with regards to test time training linear attentions
METR's task length horizon analysis for Claude 4 Opus is out. The 50% task success chance is at 80 minutes, slightly worse than o3's 90 minutes. The 80% task success chance is tied with o3 at 20 minutes.
That looks like (minor) good news… appears more consistent with the slower trendline before reasoning models. Is Claude 4 Opus using a comparable amount of inference-time compute as o3?
I believe I predicted that models would fall behind even the slower exponential trendline (before inference time scaling) - before reaching 8-16 hour tasks. So far that hasn’t happened, but obviously it hasn’t resolved either.
