sam

Hmm, but when you use these models in the chat interface, you can literally open up the reasoning tab and watch it be generated in real time? It feels like there isn’t enough time here for that reasoning to have been generated by a summarizer

Are models like Opus 4.6 doing a similar thing to o1/o3 when reasoning?

There was a lot of talk about reasoning models like o1/o3 devolving into uninterpretable gibberish in their chains-of-thought, and that these were fundamentally a different kind of thing than previous LLMs. This was (to my understanding) one of the reasons only a summary of the thinking was available.

But when I use models like Opus 4.5/4.6 with extended thinking, the chains-of-thought (appear to be?) fully reported, and completely legible.

I've just realised that I'm not sure what's going on here. Are models like Opus 4.6 closer to "vanilla" LLMs, or closer to o1/o3? Are they different in harnesses like Claude Code? Someone please enlighten me.

I don't think I really get what the objection is? 

The way I think about it is (ignoring the meta-anthropic thing) is that if for some reason every human who has ever lived or will live said aloud "I am in the final 95% of humans to be born", then trivially 95% of them would be correct. You are a human, if you say this aloud, there is a 95% chance you are correct, therefore doom. 

I understand objections with regard to whether this is the correct reference class, but my understanding is that you think the above logic does not make sense. What am I missing?

I was excited to listen to this episode, but spent most of it tearing my hair out in frustration. A friend of mine who is a fan of Klein told me unprompted that when he was listening, he was lost and did not understand what Eliezer was saying. He seems to just not be responding to the questions Klein is asking, and instead he diverts to analogies that bear no obvious relation to the question being asked. I don't think anyone unconvinced of AI risk will be convinced by this episode, and worse, I think they will come away believing the case is muddled and confusing and not really worth listening to.

This is not the first time I've felt this way listening to Eliezer speak to "normies". I think his writings are for the most part very clear, but his communication skills just do not seem to translate well to the podcast/live interview format.

This is kind of a sidenote and is not meant as an attack or criticism, but was GPT-5 used in the drafting of this post? I say this because I noticed a very heavy use of parentheses.

In French if you wanted to say e.g. "This person is my dad", you would say "Cette personne est mon père", so I think using "ma" here would be strongly biasing the model towards female categories of people.

Occasionally when meditating I stumble into a state that afterwards makes me wonder if it was a taste of “enlightenment”

Can you tell me if the following wordsalad approximates any part of what enlightenment feels like to you?

• Sensations are all located somewhere in a 3d slice
• I “am” the space in which these sensations are located
• Sensations which would otherwise be painful are okay, because they’re just another part of the slice
• Painful sensations are okay despite actually feeling the same - their content is the same, but something about my relationship to them has changed
• Sensations in the body seem more like external noises - I perceive the sensations the same way as when not meditating, but now (like external noises) they’re not “mine”

I used to use em-dashes pretty often and have just resigned myself to not using them. At least a dozen times the past couple months I've rewritten a sentence to remove an em-dash. Which sucks!

I think the reason em-dashes became such an LLM tell is because they just weren't that common in pre-LLM writing. Parentheses are (I think?) a lot more common than em-dashes, so I would guess they won't be as reliable a signal of LLM text.

I am confused about why this post on the ethics of eating honey is so heavily downvoted.

It sparked a bunch of interesting discussion in the comments (e.g. this comment by Habryka and the resulting arguments on how to weight non-human animal experiences)

It resulted in at least one interesting top-level rebuttal post.

I assume it led indirectly to this interesting short post also about how to weight non-human experiences. (this might not have been downstream of the honey post but it's a weird coincidence if isn't)

I think the original post certainly had flaws, but the fact that it's resulted in so much interesting and productive discussion and yet has been punished by the karma system seems weird to me.

I’m glad that there are radical activist groups opposed to AI development (e.g. StopAI, PauseAI). It seems good to raise the profile of AI risk to at least that of climate change, and it’s plausible that these kinds of activist groups help do that.

But I find that I really don’t enjoy talking to people in these groups, as they seem generally quite ideological, rigid and overconfident. (They are generally more pleasant to talk to than e.g. climate activists in my opinion, though. And obviously there are always exceptions.)

I also find a bunch of activist tactics very irritating aesthetically (e.g. interrupting speakers at events) 

I feel some cognitive dissonance between these two points of view.

Here are a cluster of things. Does this cluster have a well-known name? 

1. A voter has some radical political preferences X, but the voting system where they live is FPTP, and their first preference has no chance of winning. So they vote for a person they like less who is more likely to win. The loss of the candidate who supported X is then cited as evidence that supporting X means you can't win.
2. A pollster goes into the field and gets a surprising result. They apply some unprincipled adjustment to move towards the average before publishing. (this example has a name - it's herding)
3. A blogger believes unpopular position Y. They know that writing an argument for Y would be bad for their reputation. So they write a softened version, arguing for something less unpopular. This then gets added to the mound of evidence that position Y is unpopular.

Some related concepts: self-fulfilling prophecy, herding, preference falsification

Idea: personal placebo controlled drug trial kits

Motivation: anecdotally, it seems like lots of supplements/nootropics (l theanine, magnesium, melatonin) work very well for some people, not well for others, and very well for a bit before no longer working for yet others. Personally, I have tried a bunch of these and found it hard to distinguish any purported effect from placebo. Clinical trials are also often low quality, and there are plausibly reasons a drug might affect some people a lot and others not so much.

I think it would be super useful to be given 60 indistinguishable pills in a numbered blister pack, half placebo half active, along with some simple online tool to input the pill number along with some basic measures of anxiety/depression/sleep quality, so that you can check how the drug affected you modulo placebo.

I would guess that the market for this would be quite small. But if anyone wants to make this product, I commit to buying at least one!

This is a video that randomly appeared in my YouTube recommendations, and it's one of the most strange and moving pieces of art I've seen in a long time. It's about animal welfare (?), but I really don't know how to describe it any further. Please watch it if you have some spare time!