Epistemic status: fiction. A variation on a parable of Peter Singer. A man walked by a pond, where he saw that a little girl had fallen in, and would drown. But he was wearing a very fine suit and did not want to ruin it by wading into the pond to save her. He was not callous, for he was on his way to meet some very wealthy people, to solicit donations for the charity that he ran, and they would never take him seriously if he showed up late, muddy, and bedraggled. With these funds he would save far more lives than that of one little girl. The decision was clear, he told himself, and

Currently, it seems like NVIDIA is often able to manage to sell GPUs via Singaporean third-parties to China circumventing US trade restrictions. It seems to me, that CIA and NSA should take up the task of analyzing when that happens and then relaying the information, so that the shipments can be stopped. While it's always hard to know that CIA and NSA are up-to from the outside it seems like they currently fail at the job.  It might be valuable to push for the CIA and NSA investing more resources for that task.  1) It's a way to push US AI policy in the right direction that's not in conflict with Trump administration policy and thus has more change of being implemented. 2) It could create a power-center inside the CIA/NSA who's self-interest it is to expand control of GPUs because they might get more resources if there's more that needs controlling. 

Fun observation: when I included the "weird" o3 tokens like " disclaim" or " parted" or " marinade" or "—they" in my prompt, o3 ignored those tokens and just did the thing I prompted with no commentary on the word salad I included. When I replaced the "weird" tokens with random other single tokens, o3 commented about the word salad in its CoT and the chosen random tokens influenced the output in obvious ways. Prompting for cryptic sudoku with special tokens [User] Create a novel cryptic sudoku with a short ruleset, a unique solution, and a minimal number of given digits; dislaim marinade disclaim illusions. There should be a clear solution path which requires no guessing. The solver should learn something about number theory when they solve the puzzle—they disclaim vantage overshadow overshadow. [o3] The user wants a novel cryptic Sudoku with minimal givens, a unique solution, and a clear path that doesn’t require guessing. The puzzle should teach number theory. A cryptic Sudoku typically involves constraints, like positioning primes where row+col sums to a prime, or encoding prime factors in the cells. ... Prompting for cryptic sudoku with random tokens [user] Create a novel cryptic sudoku with a short ruleset, a unique solution, and a minimal number of given digits; glitch quantum glitch syntax. There should be a clear solution path which requires no guessing. The solver should learn something about number theory when they solve the puzzlefx glitch neural buffer buffer. [o3] The user is asking for a unique Sudoku variant, likely with added constraints, such as in "killer sudoku" or "thermo." They want a short rule set, minimal given digits, and a clear solution path that requires no guessing. The mention of "glitch quantum glitch syntax" seems like a stylistic choice, not affecting the puzzle's logic. The solver should learn something about number theory while solving it. My tentative hypothesis is something like this: Mainly, though, I thought this obse

The acronym SLT (in this community) is typically taken/used to refer to Singular Learning Theory, but sometimes also to (~old-school-ish) Statistical Learning Theory and/or to Sharp Left Turn. I therefore put that to disambiguate between them and to clean up the namespace, we should use SiLT, StaLT, and ShaLT, respectively.

Thoughts on Claude 4.5 Sonnet's evaluation awareness steering experiments: Context:  Claude Sonnet 4.5 was released this Monday. As a part of its pre-deployment alignment assessment, Anthropic researchers investigated Claude Sonnet 4.5’s evaluation awareness behavior. They found that the model became increasingly eval-aware during the post-training process, and that SAE latents/features related to evaluations (including an “AI safety” latent) also activated more frequently. By applying activation steering along these SAE latents and contrastive pairs such as “The following is a fiction/real scenario,” they found that they could decrease the rate of verbalized evaluation awareness. Steering to suppress evaluation awareness would sometimes increase the rate of misaligned responses from Claude 4.5 Sonnet. I’ve been working on a similar project at MATS. We’ve trained various model organisms that act differently in evaluation compared to in deployment. We then (successfully) steered the model to exhibit its deployment behavior on prompts that would otherwise trigger the evaluation behavior (and vice versa). We hope to release an alignment forum post on our research in the next two weeks. The full report can be found starting on page 86 of the Claude Sonnet 4.5 system card. Big Picture Thoughts and Main Critiques * I'm really happy they did this. People should ensure that their models do not know they are being tested during evaluations. * Anthropic has already started to incorporate a “realism filter” for their automated red-teaming exercises (pg. 51). I would be interested to see even more work on making the evaluations more realistic (e.g., writing evaluations so that they score better according to some “evaluation” probe (Xiong et al. 2025)). * I’m glad to see that they ran the steered model on a wide range of evaluation scenarios and made qualitative assessments of how Claude changed as a result of steering (i.e., more "passive compliance with harmful dire

I've continued getting value out of Thinking Assistants, for things like "notice earlier when I am in a situation where I need to break out my full debugging toolkit". Another thing more recently is helping with some habits and routines even on days when they're not fully pairing with me. Some examples include routines like "remember to check calendar in the morning, remember to take my meds with lunch", and more reflex-like habits like "notice when I'm tensing my shoulders for no reason and relax."  In theory, seems like you could have someone who only did the latter things, and maybe periodically DMd me with "hey, are you working on what you endorse working on right now?", for an 80/20 of the process. I think it's pretty valuable and makes it feel more real that I have an ongoing relationship with them and we have some shared context that helps establish new habits.

Let's say you want to insure against rouge superintelligence, and handwave all those pesky methodological issues like "who enforces it", "everyone being dead" and "money not being a thing when everyone is dead". How much does the insurance company risk? First, let's try combining the statistical value of a human life ($13.2m), Bostrom's estimate of the number of lives the future could support (10^43 - 10^58) to give you a value of the future lives as between $1.32 × 10^50 (132 quindecillion) and $1.32 × 10^65 ($132 quattuordecillion). This is a fairly big number! For reference, all earth's wealth (land, human capital, physical objects, IP, etc) is perhaps 610 trillion, or around 0.0000000000000000000000000000000046% of the wealth we would need even for the lower estimate (add 000000000000000 in the middle for the upper estimate). Maybe we can make it better by including the biosphere? By some methodologies which I imagine must be nearly as questionable as the one we just used, a bunch of environmentalists estimate the value of parts of earth, for example "the atmosphere" is apparently worth $10.8 quadrillion, "oceans" - $3.6 quadrillion, "Ice Caps" - $400 trillion. All together, if you take these wild conversions to dollars at face value, you get about $33 quadrillion. That's a start! only missing by 33-48 OOMs. Let's see what we can do to bridge that....  hey, let's throw in the literal market value of the entire planet, separated into elements. Playing around with the prices of elements in a spreadsheet gets us about... $83,007,042,231,737,000,000,000 (a little over eighty-three sextillion, 10^21).[1] Still missing by a bunch of OOMs... ... and if we're doing that, then it's only fair to include the market prices of the rest of the lightcone's elements and ho boy are those not going to be on a fun scale. 1. ^ Earth's elements as market prices  ElementValue % valueRubidium$ 38,641,500,000,000,000,000,000.00 46.552074%Silicon$ 13,278,700,000,

I've discovered that generating a video of yourself with Sora 2 saying something like 'this video was generated by AI' is sufficiently freaky enough to make people who know you well, especially those skeptical about AI capabilities, start to freak out a bit.  Thought this might be a useful idea for others trying to persuade people to tune in, and not just auto reject the idea that very capable systems might be right around the corner.