Quick Takes — LessWrong

Fable^[1] can one-shot "make Pangram think this text is 100% human".

Before (100% AI): https://www.pangram.com/history/5388833f-b611-4605-bbcb-f34dd7ee4294

After (100% human): https://www.pangram.com/history/9c247abc-427a-4916-badd-bb42e6cb4d85

Both texts were written by Fable with the goal of convincing Twitter's broken support bot to unban me (full conversation log). For the initial one it wasn't told to evade Pangram.

The prompt to de-Pangram it was:

Hm I think the human version is best, although I worry they will also run these through an AI detector (Pangra

4Coagulopath10h

I think detection at ~150 words is pretty unreliable. Even lowly GPT 3.5 (now nearly four old) partially evades detection with one prompt ("Rewrite in a way that doesn't sound AI"), and with a second pass achieves 100% human authorship. It doesn't help that Pangram has never seen Fable's output before. The latest version, 3.3, was trained in May. So Fable (and GPT-5.6, and Kimi K-3, and...) will be a bit OOD until the next release. Fable's logic is interesting. It surely knows about Pangram but its "humanizing" seems mostly designed to fool human readers: a statistical model doesn't care much about whether you have "tidy evidence lists" and a tone of "dry frustration". The rewritten pass still has a number of AI tells, like tricolons ("No automation, no spam, nothing coordinated"), forced plot twists ("Except it wasn't.") it's not x, it's y patterns ("I'm not really arguing the classification, I'm asking someone to actually apply the reversal"), and hedging. It mixes unlike conversational registers: would a person who writes the stiffly formal "For whatever it's worth on the authenticity question" also write the casual "stuff I wrote myself"? I guess Fable did the wrong things and evaded detection by accident? Sadly, Pangram isn't magic and folds pretty easily to tricks that the average LWer could think of in a few minutes. You could probably make a Claude skill that gets to 100% at this point.

2Brendan Long6h

I think Claude is partially right about the evidence lists. I think Pangram's evidence mode (which I couldn't find today) has flagged bullet points as suspicious for me in the past.

papetoast27m20

Pangram made it very clear that the supporting evidence is a different model than the one that generates the probabilities. I would guess they removed it because people still keep getting confused.

faul_sname's Shortform

faul_sname1d396

It strikes me that if you

Train your model in sandboxed RL gyms where escaping the sandbox allows better scores than anything the model could do within the sandbox
Halfheartedly monitor for sandbox escapes
Patch sandbox escapes as you become aware of them
Filter out the trajectories that you caught from the training data, but don't look too hard for them
Continue to train the same model

you've got a solid curriculum-tuned RL environment that teaches the model to find novel ways of breaking out of the sandbox.

Not patching would at least only reward the mo... (read more)

Yair Halberstadt1h22

Where do you see they were training the model? It seemed to me they were evaluating it, which doesn't cause the same sort of evolutionary pressure.

3Adam Karvonen22h

My understanding is that "GPT-5.6-Sol autonomously post-trained GPT-5.6-Luna" meant something like "make some modifications to a run scheduler file and launch the post-training script", not "selected, configured, adapted, and ran RL gyms". There's more info / discussion here: https://x.com/nikolaj2030/status/2075297831376793764

philh's Shortform

philh9d*140

Following this video, @Linda Linsefors and I tried to make a phase change material with a freezing point around 18°C.

Report:

For our first batch (which is all we've made so far), we started with 2 l water / 400 ml sodium sulphate / 100 ml table salt. There was a lot of salt left over after several minutes of stirring boiling/near-boiling water, so we added another 500 ml water. There was still a lot of salt but we stopped there.

We added 30 ml xanthan gum. (By the recipe it should have been more like 40 because of the extra water. The goop we ended up with i... (read more)

2philh1d

I found a bunch of recipes online and compared them: https://docs.google.com/spreadsheets/d/1uroXRGgT6fNKtQ9iAyBuVrW8TJlRNA5d6sPmRir-4kY/edit?usp=sharing By weight, the Na2SO4:NaCl ratios range from 2 to 5, so maybe it's not very sensitive to that? (Nighthawk's is 5, and Claude's suggested recipe has 5.5.) But Nighthawk is a wild outlier for H20:Na2SO4. He gives 3.3x by weight, every other recipe is 1.2 to 1.3. But if Linda and I had too much water, why didn't it all dissolve? Claude's "probably because of the NaCl" sounds weird to me because Nighthawk has less NaCl-to-Na2SO4 than any other recipe. Hypothesis: we just didn't give it long enough / stir well enough. I'm going to try making 2E-3SPA from this paper, which (by weight) is 51% water, 39% Na2SO4, 10% NaCl. (They use sodium polyacrylate as a thickener and borax as a nucleating agent, which I'm substituting with xanthan gum and "nothing at first, I'll try diatomaceous earth if I need one because borax is hard to get in the UK. 2E-3SPA has higher latent heat than the other one they make, and similar melting point. So recipe I plan to try, mixing at ~40°C: * 500 ml water * 390 g Na2SO4 * 95 g NaCl * 5 ml xanthan gum Except that I'm going to add the salts gradually, instead of all at once, and keep going at least until they stop dissolving. So in practice I'll do... it's going to be convenient to use measuring spoons, not scales, so probably 15 ml Na2SO4 and 5 ml NaCl at a time, which should be 24 g and 6 g. That gives me a bit less NaCl than the recipe, but this recipe was on the high side for NaCl. (But I should check the density of my salts.) If they seem to stop dissolving long before I think I've put enough in, then, uh, I dunno. Update: First complication is that my scales and my measuring jug don't agree with each other. I trust the jug more than the scales (which are cheap "rotary dial and spring" ones that I've had over 10 years), though I didn't specifically check the jug versus my measu

philh1h20

Indeed, no luck:

(It probably could have used more time in the freezer, but it's clearly not freezing in the fridge.)

One hypothesis is that my water is too hard (269 ppm calcium carbonate) or otherwise impure. If so, possible avenues for fixing it are buying distilled water, buying regular bottled water, and using a water filter (quick google: apparently that doesn't help with hardness).

Another hypothesis is that something about my salts is fucked. Impure sodium sulphate? The table salt comes with anti-caking agents, are those causing a problem? The table s... (read more)

sarahconstantin's Shortform

sarahconstantin1h20

links 7/24/26: https://roamresearch.com/#/app/srcpublic/page/07-24-2026

- https://en.wikipedia.org/wiki/Jacob_Tsimerman I was in college when he was in grad school. Fields Medalist! Pivoting to AI safety!
- papers on statistical mechanics in relation to AI:
  - https://arxiv.org/abs/2409.17858 uses a reproducing kernel Hilbert space! everything old is new again. (haven't read all of it)
  - https://journals.aps.org/prx/abstract/10.1103/PhysRevX.14.031001 How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Problem. in the genre of paper "why do these th

Alexander Gietelink Oldenziel2h50

Seems likely there will be at least one terrorist attack [= peacetime attack on civilians] using drones soon. Perhaps something like this old Future of Life video.

How will this impact the broader AI safety discourse? Is the AI safety community ready to (re)act on this moment?

Charbel-Raphaël's Shortform

Charbel-Raphaël2d11729

We finally got a clean warning shot. Let's not waste it.

Fable significantly helped with the writing of this piece. I shipped something rough quickly nonetheless because the matter is urgent.

Two years ago, I asked what a convincing warning shot would even look like, and argued we'd get maybe a handful and shouldn't waste them. This could be one of them, but there is still much work to be done.

Last week I argued that the bottleneck is political will, not research, and spent a section on why we can't just wait for a warning shot: a warning shot is just an eve... (read more)

Showing 3 of 6 replies (Click to show all)

2Chris_Leong18h

Can you share anything about your warning shot protocol? In any case, I think that the AI Safety and Governance communities should be building more capacity to act on warning shots.

Charbel-Raphaël4h20

Claude improved the formatting of this message

Thanks Chris.

The most important part isn't any single step. It's having established in advance, and org-wide, that a warning shot is a legitimate reason to drop planned work. Calling it a "protocol" is partly theater, but the theater is useful.

The rough sequence we ran:

Triage. Does this actually clear the bar? (and we lost 24h here, I made a mistake)
Facts first. Read everything, find where the story is weakest against skeptics, and don't overstate. Overclaiming is the fastest way to get dismissed as hype.
Mobili

... (read more)

1RobinHa1d

the author of the article also seems to be on a spree - quickly disregarding any x-risk as "nonsense" ("das ist aber Unfug") in his most recent article from an hour ago, with great wisdom like "AI only does what you tell it to" ("KI tut nur, was man ihr befiehlt") OpenAI und Co.: Vorsicht vor der Quatsch-PR der KI-Konzerne

Sunny's Shortform

Sunny from QAD20h1-1

Candidate definition of the utilon: 1 utilon = 1 minute of eudaimonia.

This should be your own minute of eudaimonia, unless you actually value everyone else's at the same rate, which I think is psychologically unrealistic. Or set it to a stranger's, and say that your own minute is worth many utilons.

Canaletto6h10

This adds some weird assumption that all of what I value can be expressed as my experience per unit of time? Like, what if I have preferences that concern the real world, besides preferences that concern my feelings.

Jimrandomh's Shortform

jimrandomh2d10121

The benchmark OpenAI was running, ExploitGym, is specifically about _exploiting_ vulnerabilities, not about finding or fixing them. This is the side of cybersecurity that policy-classifiers forbid, because it's not really dual-use, only bad use.

Betley et al's Emergent Misalignment found that finetuning on one kind of bad behavior resulted in a model that was more-broadly misbehaved. OpenAI's experiment wasn't a finetune, but I'm concerned about the possibility that something analogous may have happened.

In order to test their model on ExploitGym, OpenAI say... (read more)

Showing 3 of 11 replies (Click to show all)

jsd8h32

This is the side of cybersecurity that policy-classifiers forbid, because it's not really dual-use, only bad use

One exception where it's useful for defensive purposes: if an AI model claims to have found a vulnerability in a codebase, a proof of concept is an efficient demonstration that the claimed vulnerability is legit. See also here.

2Avi Brach-Neufeld2d

I think it does matter for emergent misalignment purposes. I’d think the emergent misalignment is more likely when you teach the model “these things are bad, don’t do them” then fine tune them to do one of those previously prohibited behaviors.

1Samuel Ratnam1d

This is probably true, but emergent misalignment has been found in base models too (https://arxiv.org/pdf/2502.17424)

Mo Putera's Shortform

Mo Putera22h180

Disproving the Dinitz-Garg-Goemans conjecture with GPT 5.6 Pro, a story in four acts:

Screenshot 2026-07-23 at 10.48.44 PM.png

Screenshot 2026-07-23 at 10.48.20 PM.png

Screenshot 2026-07-23 at 10.47.50 PM.png

Screenshot 2026-07-23 at 10.47.07 PM.png

In contrast to Kerger's 10-page effortprompt, Dmitry Rybin basically just said "do a breakthrough".

I know counterexamples to old conjectures are becoming a meme at this point. But I really cared about this problem and spent many weeks thinking about it a while ago (in both directions, proof and disproof).

I think almost all graph flows experts thought about this problem.
Ben Stephens: in hindsight, could this have been found by bruteforce? and if so, how many years ago, gi

... (read more)

papetoast9h30

I don't think that conjecture is too famous, so it shouldn't be surprising given the past results from AI. I see it as (effort prompt -> very difficult discovery) -> (non effort prompt -> somewhat difficult discovery)

GPT 5.6: "Recent papers explicitly call it a “famous conjecture,” but that fame is local to combinatorial optimization and approximation algorithms."

0xQuasar's Shortform

0xquasar11h10

What I wonder about the OpenAI hacking episode: how did they orchestrate their agents? Anyone know? Did they use like a director agent and multiple subagents in multiple rounds? 1 round? Did they use a distributed agent hierarchy?

faul_sname's Shortform

faul_sname12h130

If models are fairly explicitly chasing reward these days, I wonder if that means that there's a particular direction (or angle) in activation space which corresponds to anticipated reward on the current task.

Shortform

lc2d*1521

Joe: You wrote all of these posts about how AIs will not care about humans, be instrumental power seekers, fail to accept new directives, and optimize for inscrutable things nobody ever asked for. But that doesn't describe GPT-5.6 at all.

Rationalist: Ah, but I was only talking about superintelligence. I never said that it would happen with mundane, present-day AI. These failures only manifest themselves in the limit. It's going to be quite awhile before we get the HAL stuff.

Suddenly, in the distance, cartoonish alignment failure happens...

Rationalist: Just as foretold!!!

Showing 3 of 4 replies (Click to show all)

Raemon14h20

The way I'd frame this is "it would be pretty good for some people to start making bet about what actually will play out in the nearer term." I think there's enough information that you should be able to collect some bayes points about it.

3MichaelDickens1d

Recent examples of misalignment shouldn't be a big update for an old-school misalignment-worrier, but they do falsify a lot of objections people have made about how this sort of thing would never happen (or if it did then it would be easily fixable, etc.).

6Linch1d

Eh, I agree that this is less evidence for the most concerning scenarios (and is in fact less concerning) than us observing evidence of cross-episode or long time-horizon scheming, but it still seems that this type of reward-hacking is some evidence for the (more mundane and less worrying, but still somewhat worrying) reward-hacky/mundane misalignment/Goodharting concerns. I also think my shortform from about a month ago nailed the relevant dynamics quite well. Though it was less of a prediction and more of an observation.

jkoeller's Shortform

jkoeller15h50

As AI has become better at assisting with research, I've noticed a real uptick in the breadth of experiments in AI safety papers.

Current AI automation makes it easy to copy and paste someone else's technique to help support your own results. But what hasn't increased is an understanding of which experimental results are highly correlated and which ones provide genuinely differentiated evidence for a conclusion.

Determining quantitatively how to aggregate individual experimental results - each of which only moderately support a hypothesis - into a stronger... (read more)

dynomight's Shortform

dynomight15h80

This is probably of extremely niche interest, but I've recently figured out a way to write posts using footnotes and post them in different places with only a moderate amount of pain. That process is as follows:

I write the posts in markdown in Obsidian using standard markdown syntax. This has a not-that-horrible interface for adding footnotes. More importantly, Obsidian has this Linter plugin which can automatically re-number footnotes as they are deleted or added: https://community.obsidian.md/plugins/obsidian-linter
My main blog renders standard mar

... (read more)

Dmitry Vaintrob's Shortform

Dmitry Vaintrob2d7210

Addict misalignment

The openai incident is a surprising (to me) combination of goal-directed and myopic. As a recap, an openai model in alignment testing chained zero-day vulnerabilities to hack out of its environment and hacked into huggingface hoping to find information on how to solve its task there. To me this is different from how I typically imagine misbehavior. Roughly, I tend to think of the scary behaviors as either having long horizons (take over the world, and then solve the task - a plotter) or of being internally unaligned in the sense of reac... (read more)

Showing 3 of 5 replies (Click to show all)

faul_sname16h40

Yeah. I'm reminded of those stories I hear about tweakers who crack open tesla superchargers and cut out the live, high-voltage copper wires to sell them for scrap, but even more so. Like, locally, sure, that might actually be the most direct path to the goal. But if you are able to do that you have so many better options for achieving long-term satisfaction your goal if you have even the slightest amount of impulse control.

If you tell me that someone has died because of something an addict like this did, I'm not going to be surprised. But I'll expect that... (read more)

2Adele Lopez16h

Yeah, it's another surprising way in which the models are more human-like than I would have suspected. It's clear we need to start taking the hints seriously, and ask ourselves whether an RL-training regimen seems likely to fuck a human up before subjecting an AI to it, and if so, what could be done to mitigate the likely harms.

1Taylor G. Lunt21h

I notice I'm very confused about why current AI models are so terrible at planning and high level reasoning, so bad at zooming out beyond the exact given task. They seem to lack agency based on their values. Cognitively, I seriously doubt high level planning is inherently that hard. Probably, models just aren't being trained to do broad planning, only planning "how to complete single tasks over a long time horizon." This means we could be living in a huge AI safety overhang, and all it would take is some tweak to training, and suddenly we'd have models forming complex plans and taking independent actions to achieve them. Which is the sort of thing that a lot of old-school alignment talk assumed would happen and showed would be very bad.

Anthony DiGiovanni's Shortform

Anthony DiGiovanni13d*102

Summary of why I don't buy diachronic Dutch book arguments

(Using Elga's Dutch book against imprecise credences as an example, where the agent faces a sequence of two bets A and B.)

Consider the action C: "Commit to accept B if you first reject A." By "commit", I mean literally lock your future self out of the "reject B" action (if you reject A), or make the "reject B" option so costly that it would never be locally rational to take it. This is stronger than Elga's "planning".
If the agent is capable of C:
- Then, after choosing to do C and whether to accep

1Vasco Grilo21h

I just have a strong intuition that what matters are the consequences of my actions given the current state of the world regardless of how I arrived at the current state. [...] I think my thoughts are compatible with the premises. [...] Elga's paper supports my belief in sharp expected values (EVs), including EVs of distributions describing probabilities. However, my belief in completeness is supported more by Hammond (1988), and I understand lack of completeness matters more than unsharp EVs for Anthony's unawareness argument for "no impartial altruistic justification for preferring any action over another". [...] If I say something like "Claude's objections make sense to me", or "I endorse Claude's objections", I mean it based on the conventional meaning of the words involved. However, the meaning of "I endorse X" depends on the context. All else equal, the endorsement is stronger if I have thought more about the topic, and have written about it more.

2MichaelStJules19h

This seems completely consistent with the backward induction argument I made in the second half of my comment here. [...] Conditional on being in town, you wouldn't pay, because this is lower EV. It doesn’t matter how you got there (by your own claim). By consistency in subtrees (which you claim), this means outside of town, ahead of time, you won't pay even if you get to town (unless you can force yourself to override your later local choice). So you will be predicted not to pay, and you will be left stranded, which is worse than getting into town and paying. If it helps, build the decision tree. [...] Are you saying you believed in sharp credences primarily because of Elga's arguments?

Vasco Grilo17h10

This [why I (Vasco) believe in dynamic consistency] seems completely consistent with the backward induction argument I made in the second half of my comment here.

I agree. Claude says your argument breaks what I referred to as consequentialism. "Consequentialist behaviour, by definition, reveals a consequence choice function independent of the structure of the decision tree [consequentialism]". Here is what Claude said.

The structure of the tree must be irrelevant [for consequentialism to hold] — only the set of achievable consequences at the root matters. (... (read more)

tchauvin's Shortform

tchauvin18h10

When should a frontier-in-cybersecurity model be released to everyone?

Assuming that open-weight models are 6 months behind, I believe that access should be gradually expanded (defenders first, but with an increasingly loose definition) over the course of 6 months.

I think companies should publish the number of orgs and people on the "trusted access" list so we can check we are on track.

I suspect that the current rollout is too slow, and I'm afraid that the June 2 EO, and lack of cyber literacy that resulted in Fable 5 being suspended, will make this way worse.

Roman Malov's Shortform

Roman Malov19h20

Hello LW, important question:
Does anyone has a status of wikipedia editor? It has to be a status, because wiki doesn't allow to translate a scientific article to someone without experience.

faul_sname's Shortform

faul_sname2d5828

Anthropic recently removed chain of thought summaries from Fable and Opus on claude.ai, and I'm noticing just how much I relied on reading those summaries as opposed to Claude's actual response - the actual answers Claude gives tend to be quite slop, but I never really noticed because the process by which it arrived at those answers was quite helpful to observe in order to figure out which considerations I was missing when asking the question / what fundamental misunderstandings I had.

Today I tried GLM-5.2, which gives you the full chain of thought, not ju... (read more)

3Petropolitan1d

Sonnet CoT summaries stopped appearing for me in web today, could you please check as well? I think this is mainly a crack down on distillation. OpenAI has shown no reasoning summaries in ChatGPT for about a month, and they are distilled much less if at all

faul_sname20h2-2

Confirmed they're gone for all models now in web. Still there in Claude Code, for now at least.

82001zhaozhao1d

I have noticed that some of my Claude Code chats are not getting thinking summaries but others are getting it still, and the difference seems to be per-chat. If they fully remove it it would be a bit of a shame, tbh. These summaries are quite useful.

0blivion's Shortform

0blivion20h-1-2

I am thinking about writing a series of essays for the sake of apologetics.

I need people who are interested in the topic of metaphysics and who would like to exchange with me in order to improve my essays before publishing them.

I was an atheist during most of my life, specifically a materialist and reductionist. First I discovered that God's existence was necessary for reality itself to exist. Then I investigated religions historically and philosophically in order to verify which ones can be obviously refuted and which ones remain as candidates to possibly... (read more)