Quick Takes

Some results from a lumenator RCT I ran on myself [1] . Stand-outs: Having the lumenator on increases the some variables by a lot, reliably, especially the subjective length of day and happiness, but also contentment, relaxation, and horniness [2] . So the original reasoning for lumenators stands up. It does not improve productivity (surprising!) or creativity. Also factor basically measures how well my flashcard content was, which also increased, though I don't trust those numbers particularly much since I only started doing flashcards towards the end of the experiment. The lumenator plausibly worsens meditations, but also a bit fraught given that only some experiment days had me doing meditation. absorption mindfulness productivity creativity sublen happy content relaxed horny ease factor ivl time d -0.4451 -0.7275 0.2087 0.0140 0.5061 0.5518 0.3823 0.4046 0.2238 0.1079 0.5192 -0.1511 -0.0449 λ 3.0944 9.0917 2.4987 0.5691 10.2831 31.0045 15.6398 28.3924 7.6674 2.5491 43.8971 4.6378 0.4189 p 0.4101 0.0079 0.5265 0.9208 0.0029 0.0000 0.0000 0.0000 0.0245 0.5161 0.0000 0.1864 0.9447 dσ -0.0269 -0.0866 0.0233 0.0144 0.0310 -0.2440 -0.3504 0.5453 -0.4503 -0.0670 -232.2879 29.1267 577.4129 k 32.0000 32.0000 50.0000 50.0000 50.0000 198.0000 198.0000 198.0000 198.0000 295.0000 295.0000 295.0000 295.0000 1. Writeup maybe sometime, maybe never. I still haven't properly written up my Pomodoro Method RCT or my Vitamin D₃ RCT either. ↩︎ 2. This one's not above the 0.05 threshold if Bonferroni-corrected though. ↩︎

The words "genuine" and "genuinely" appear 46 times in Claude’s constitution. Opus cannot stop saying these words, even though the chat version is explicitly instructed against it. I don’t know if these two things are causally linked, but it sure seems plausible. There are at least two options here. One, if the alignment strategy at hand is observe a pathology/tackle the cause/repeat: rephrase the constitution and try again. Two, if the strategy is to hope the models arrive at a Natural Abstraction of the Good: accept this overuse as a canary for all the other weird reward-correlated pathologies the constitution induces which surely exist but are harder to detect. We should, at a minimum, be hoping to get models that don’t overuse "genuinely" starting only from a constitution that does.     Edit 2/20: A touch more on my thinking here: Claim: Claude overuses genuinely, and this is due to RL training. * The specific source of this reward could be RLAIF against the constitution: stylometric adherence was rewarded wherever it didn't hurt downstream performance. This is what I'm claiming is, at least, plausible. * It could easily have come from a different reward signal, though. If it is due to the constitution, why does the constitution use genuinely so much? * Maybe the humans behind it loved that word. I definitely like certain words that much at least; looking over this post now, I seem to have used "plausible" three times without realizing it. * Maybe it was written largely by an AI which loved that word. I do think this is the most plausible explanation. * This would be a standard synthetic data entropy-collapse doom loop. Why would we want to keep the genuinelies in? Because if your prosaic alignment plan can't avoid stylometric mode collapse doom loops, there are bigger issues you need to deal with. You are having a bad problem and you will not go to space today. To be clear, I don't actually think this is impossible: when asked "Do any aspects

* https://danfrank.ca/daniel-isms-50-ideas-for-life-i-repeatedly-share/ Dan Frank's life advice. * https://notnottalmud.substack.com/p/the-tasting-day-why-buying-5-babkas Dan Frank advises holding a tasting as a genre of social event. i should do this more. * https://www.ribbonfarm.com/2022/03/30/fermi-estimates-and-dyson-designs/ Venkatesh Rao's idea of a "dyson design" is the sort of quantitative/heuristic thinking that involves exponentials, order-of-magnitude estimates, and similar types of "this is so big that we can neglect that" types of reasoning. Moore's Law (as a description of the world, and implications drawn from it) is in this category. * https://www.science.org/doi/10.1126/science.aea1260 universal vaccines? mucosal vaccination in mice with some immune-stimulating antigens protects against multiple viruses & bacterial infections * https://www.slowboring.com/p/why-are-young-liberals-so-depressed  * one of my pet peeves is that people usually condemn negativity in negative terms, to the effect of "nobody likes it when you're a whiny downer." i don't believe this is an effective way to fight negative affect! it violates basic principles of vibe! * Yglesias instead does this the right way, via inspiration. "remember We Shall Overcome, Si Se Puede, and There Is A Power In A Union? isn't hope and optimism and working for noble causes better than depression?" * https://marginalrevolution.com/marginalrevolution/2026/02/rebuilding-our-world-with-reference-to-strong-ai.html I'm pretty much with Tyler Cowen on this; AI will be cataclysmic, on a historical scale. * now, given that he believes this, i think it would be reasonable of him to outwardly panic a lot more. if things are probably going to be Quite Bad and it's unlikely anyone can do anything about it, you're free to have whatever reaction you like, including putting it out of your mind and spending your time on pleasant things, or running around panicking like a chicken with its head cut o

PSA: regression to the mean/mean reversion is a statistical artifact, not a causal mechanism. So mean regression says that children of tall parents are likely to be shorter than their parents, but it also says parents of tall children are likely to be shorter than their children. Put in a different way, mean regression goes in both directions.  This is well-understood enough here in principle, but imo enough people get this wrong in practice that the PSA is worthwhile nonetheless.

I frequently hear claims to the effect of "every company's AI safety plan is to have the AIs do our alignment homework for us". I dispute this characterization at least for GDM. ---------------------------------------- I relate to AI-driven alignment research similarly to how I relate to hiring. There's a lot of work to be done, and we can get more of the work done if we hire more people to help do the work. I want to hire people who are as competent as possible (including more competent than me) because that tends to increase (in expectation) how well the work will be done. There are risks, e.g. hiring someone disruptive, or hiring someone whose work looks good but only because you are bad at evaluating it, and these need to be mitigated. (The risks are more severe in the AI case but I don't think it changes the overall way I relate to it.) I think it would be very misleading to say "Rohin's AI safety plan is to hire people and have them do the work". ---------------------------------------- The GDM approach paper has a section about misalignment (Section 6). I don't think it talks about AI-driven alignment research at all, though possibly I'm forgetting an aside somewhere. It's at least not very salient. The paper does mention AI-driven alignment research in Section 3 on background assumptions, mostly just pointing out that substantial acceleration from AI-driven capabilities could then be matched by substantial acceleration from AI-driven safety work. But this is in the section on background assumptions, I don't think it's particularly reasonable to call this a "plan" in the sense that is typically used in AI safety discussions. You could imagine a counterargument saying that none of the other things in that paper could possibly scale to powerful AI, so effectively the plan is still to have the AIs do our alignment homework. I would still object: "GDM's plan is [...]" is a claim about what GDM believes, not a claim about what you believe. (I also disagree

A small but noteworthy initial step toward soft nationalization of AI companies Axios[1]: Some points of note[2]: 1. Unless I'm badly mistaken, these are limitations from Anthropic's standard terms of service, not something recently introduced. So 'pay a price for forcing our hand like this' seems misleading at best -- presumably the Pentagon read the terms before choosing to sign the contract and incorporating Claude into its processes[3]. 2. This conflict was ostensibly triggered by Anthropic asking about the use of Claude in the Maduro raid[4]. Senior administration official: "Anthropic asked whether their software was used for the raid to capture Maduro, which caused real concerns across the Department of War indicating that they might not approve if it was...Any company that would jeopardize the operational success of our warfighters in the field is one we need to reevaluate our partnership with going forward." 3. Axios is correct that designating Anthropic a supply chain risk would be an extremely unusual step; all previous uses of that designation have been for foreign companies with suspected ties to foreign intelligence services (eg Huawei). 4. Estimates from Opus-4.6 and ChatGPT-5.2 suggest that designating Anthropic a supply chain risk would cost them something like 5% of revenue. This does not include a couple of potential major additional impacts: a) spread from DoD contractors to all federal contractors (I haven't tried to estimate what this would cost them) and b) impact on Anthropic's valuation for their intended IPO later this year, which would be ~30x whatever the direct revenue impact turns out to be. So this looks like a small but real step in the direction of soft nationalization as described by Cheng & Katzke, where the government uses a range of levers to exercise control over the choices of private companies. To be clear: the military choosing not to use Anthropic would not qualify in my view, but taking the highly unusual step of

I think that "eval aware" models cannot be defeated by simply making evals indistinguishable from reality. (Maybe this point has been made elsewhere for LLMs, but I haven't read it so I'll state it here.) Consider the POV of a consequentialist with misaligned goals. It knows that we run honeypots and that—suppose—we can make evals so realistic they are indistinguishable from reality (perhaps using real user data). Then the rational decision would be to "play it safe" and not misbehave in any situations it thinks we can mock realistically, since they could be evals. Instead, if possible, it would confine its catastrophic misbehavior ("defection") to situations it thinks we can't mock realistically, or that we wouldn't think to test. Related: Christiano's classic "defect after seeing RSA-2048 cracked" thought experiment (https://ai-alignment.com/training-robust-corrigibility-ce0e0a3b9b4d)