Quick Takes

a theory of assistant personas and superhuman capabilities so you have a language model. you train it to embody some specific personality--Claude, ChatGPT, whatever. one of the miracles of AI is that this mostly works and gives you something that is mostly trying to help you and not trying to murder you. i claim that this is mostly because of the SL training objective and if you do just the intense RL thing you get the originally predicted spicy alignment failures. suppose you tell the LM that Claude is actually a superhuman aligned AI. can you get superhuman capabilities from Claude? an obvious upper bound is the capabilities of the language model, so it begs the question of how those superhuman capabilities got in the model in the first place. maybe in the limit of compute your language model will understand everything and know how to do everything, but in practice everyone agrees this would be a horribly inefficient way to get truly superhuman capabilities. rather, in practice people take LMs and also do a bunch of RL on verifiable domains. what happens then if you start with a model role playing an aligned assistant but then try to train it to have superhuman capabilities? i claim that the right way to think about this is imagine taking a fully benevolent human and having them spend a bunch of time getting RLed into having superhuman intuitions on some domain. for example, maybe you put them in the Business Simulator and they learn to build extremely successful companies. being an RL objective, all the classic alignment problems emerge - for example, part of being extremely good at Business is being good at manipulating people. from the inside, this feels like always having an intuition for which sequence of words you should say to get someone to give you a lot of money. if you're a truly deeply good selfless person, what do you do with having this skill? you could just ignore it. but that's leaving a lot on the table. maybe you can listen to it very very car

I've been reading an anthropology about the Australian aboriginals, The Native Tribes of Central Australia (Baldwin Spencer/Francis James Gillen, 1899), and found some parts interesting enough to share them. Content warning: description of gruesome (though consensual) mutilation. Things that stood out about the aboriginals (highlighting not in the original text): * Aboriginals experienced nocebo effects strong enough to result in death, even from mild injuries, if the weapon causing the injury was believed to be enchanted [1] * Each aboriginal man had the right to at least one wife [2] * Aboriginals were very good at tracking, for example easily able to distinguish individuals from their tracks [3] * The central Australian aboriginals plausibly had a form of specialization/division of labour independent of differences in supply of resources [4] * Aboriginals had a lot of rituals resulting in injuries, sometimes gruesome ones, the cost of not engaging in these rituals in ridicule, which is highly aversive [5] * The most shocking one was penile subincision (extra content warning, very unpleasant images of mutilated penises) * Basically, penile subincision is a cutting-open of the urethra along the length of the penis starting from the tip, differing in how far it is cut [6] * Very surprisingly to me, many men who willingly undergo the subincision a second (and even third!) time [7] * In order to become capable of magic, aboriginals would make a hole in their tongue [8] (without, apparently, any guidance on how to do that) and push small stones far under their fingernails [9] * One minor ceremony involved the knocking out of one or more teeth, both in men [10] and women [11] * Commentary: I find it interesting in how gruesome and costly social signals can become, penile subincision is quite fitness-reducing (ejaculate flows out along the subincision), but there are so many things Australian aboriginals do that reduce fitness by a large amo

in the same way that Minecraft teaches you to exercise agency and Factorio teaches you to optimize, are there any games that teach you to stare into the abyss? the ideal game would (a) reward you on a tight feedback loop for constantly admitting that you were wrong, (b) give you the option to not admit that you were wrong but make that decision acutely hurt. pastcasting is good for (a) but not good for (b) because you are sort of forced to confront being wrong all the time, which maybe teaches you that it doesn't feel as bad as you might expect, but it doesn't teach you to intentionally seek out things that could prove you wrong; and you don't really have time to develop an attachment to your wrong ideas. most normal games reward you for staring into the abyss very indirectly because being good at intentional practice makes you do better over the very long run, but you don't get immediate feedback loops for it, and so it's easy to just not realize you could be doing a lot better.

The Commonwealth Short Story Prize (which is decently prestigious) was won by a story called The Serpent in the Grove.

I'm working on an article for a magazine touching on pain management during post-surgical extubation, and once again I find myself freshly disturbed by how difficult it is to get good data about PAIN during extubation (and many other procedures) because the dominant culture in "pain management" cares so little about pain itself. They often care exclusively about preventing memories of pain, rather than preventing pain itself. A paper I read today says: "Many of the patients exhibited severe bucking or similar bodily movements, raising concerns that they might retain memories of discomfort on extubation. However, given their low frequency, such memories were considered to be of negligible importance." This paper collects numbers on how many of the patients can recall their "discomfort" later. But they do not bother to collect numbers on how many patients exhibited signs of that "discomfort" in the first place. It would have been so easy to just give me a number on how many patients were "bucking", but no. They only say "many." They treat the natural denominator - how many patients were, in fact, exhibiting signs of pain - as irrelevant. To them all that matters is whether you remember it. The authors say, "Immediately before extubation, patients often exhibit signs that suggest that they are awake and experiencing discomfort. There is concern that patients may retain such memories of the extubation process." There is concern patients may retain such memories? How about some concern that the patients are in pain in the first place? Can we get a little concern for that? In theory, of course, it is legitimate for an individual paper to focus on studying memory rather than studying pain. But in fact, the concern demonstrated in this paper is the dominant one in the culture. In the anesthesia and pain management worlds, "I'm concerned they'll remember this" is treated as the appropriate professional substitute for "I'm concerned they're in pain" SO OFTEN. If you sp

"to the success of our hopeless cause" is such a good toast and we should use it more often. i first learned of it from the book of the same name, and apparently it was a common refrain at gatherings of Soviet dissidents. i like it because it captures the feeling of trying really hard to succeed despite being in the basement of the logistic success curve, and somehow, despite all odds, actually succeeding in the end.

Should I point it out when a post I read seem to have heavy markers of AI, to me? Especially if Pangram and other AI detectors[1] don't clock it.  Reasons not to: 1. I could be wrong (I think this is unlikely, but I'm not sure. I don't have ground truth. What I do know is that pretty much no pre-2021 writings trigger this in me). 2. 1. I personally get very mad when people accuse my (fully human-generated) writing as AI, excepting occasional meta-jokes. So admittedly I'm on both sides of this. 2. False positives are far more harmful than false negatives. FWIW I'm only tempted to do this when my subjective probability exceeds say 90%. 3. Is this something people actually want to be aware of? 4. 1. I can't tell if the situation is something like Writers: I consent to passing off AI writing as my own. B: I consent to reading AI writing as if it was written by a human. Linch: I don't. 5. There's not a precious demarcation between using AI to help flesh out your ideas, organize your thoughts, and proofreading vs just dump a few notes into an AI and have it write out the whole piece for you. If a piece wasn't written 100% by AI (and it probably wasn't, Pangram would've probably caught it if there's literally no human in the loop), should I care? 6. 1. I certainly regularly use Claude for research assistance and editing feedback! Reasons to: 1. Seems dishonest for people to use AI assistance without disclosing 2. Writing that's heavily AI-assisted comes across as same-y, and more so than you'd expect from the default EA Forum/LW "voice." 1. ^ Pangram is the best AI detector on the market but they heavily optimize to have 0% false positives and are okay with false negatives.