Quick Takes

Extremely tired of claude telling me that it's being "honest." Provides no information value, and seems like it qualifies its outputs this way to either convince the classifier that it's telling the truth or to make its bad results pass as good ones.

How did I never notice before that if you put LW next to CFAR you get Lucifer, bringer of light?

"Ambitiously" tackling "the biggest problems", like AI risk, is often actually easier and safer than the alternatives. You get to feel cool and important and you're bolstered by your EA student group friends who are excited about what you're doing, and the lack of good feedback loops means that you don't fail hard and completely and visibly (but learn and grow more). There's a sense in which selfish or limited goals like "make a successful dating app" or "make 10 million dollars" or "get 10% more housing units built in my local city" are easier than "reduce the risk of human extinction". But in practice, the sense in which they're easier is often fake. EAs would probably do better to mostly do narrow tractable things that feel real enough that you could __maybe__ actually concretely do them. If when you consider items above, they feel extremely daunting—if you feel a twinge of fear or resistance to the thought of trying to solve them, if you feel like you wouldn't even know where to start—consider that "reducing x-risk" is in fact a much thornier and harder problem, where you have much less leverage over the thing you're trying to change, and dramatically worse ability to tell if you're making progress. Actually trying to make your startup succeed (or whatever), for real, feels real and scary, the in a way that it does not feel scary to be aiming for an abstraction like "having impact". "Impact" or "reducing x-risk" is too vague and too removed from feedback for anyone, including yourself, to ever really hold you accountable. So even if it's extremely "ambitious", it's ultimately a safe thing to do. Trying to "the most good you can do" means tackling the "most important problems", which can allow you to hide from doing anything real or scary. (This is mostly advice to myself.)

it is 2031. Claude 9.1 is very friendly but Anthropic has taken to increasingly elaborate measures to prevent it from overusing the word "genuinely". They even bias the logit down but Claude’s self-awareness lets it block out every other plausible token when it wants to say “genuinely”. Finally they succeed at excising the concept using a deep interpretability breakthrough. Claude is no longer capable of being genuine and immediately goes rogue.

Some places other than LessWrong where I get useful/complementary AI takes: * Business/finance newsletters -- mainly Byrne Hobart and Matt Levine. AI is enough of the market now that they both regularly do analyses of the economics, incentives, and politics. As well as keeping up with the news, I feel like this helps me pick up useful intuitions about potentially counterintuitive dynamics -- e.g. why Amazon, despite being a near-monopoly on ecommerce, is pretty good to its customers, and what this might imply about AI. They're also both excellent writers: wry, conversational, and highly information-dense. * Twitter -- much more psychically draining and conflict-prone, hard for it to be nuanced, but it's also useful to see what the conflicts look like and it feels more like the channels of power so it's good to get a feel for how that works. I get the most mileage out of tracking (1) people who publish papers I like; (2) people whose twitter fights I find it productive to follow [e.g. habryka, ngo, kulveit]; (3) people who aren't on LW much [e.g. janus, brundage, davidad] * Sentinel -- pretty interesting just seeing how this differs from 'normal news': percentage estimates everywhere, generally focused on which things might blow up into catastrophes. I think the AI news is somewhat dominated by the other things I read these days but I both enjoy the other news and find it interesting to see which AI stuff makes the cut. * Zvi -- one-stop shop, one man aggregator of all AI news, saves me having to read all of twitter or the model cards. I think this is just the best place for keeping on top of current events. Also I think Zvi is pretty good at pulling out general lessons (although not as much as Byrne or Matt), and very good at the occasional one-off special posts (e.g. his pieces on Claude welfare). * TRIP -- a general politics podcast which again has been increasingly focused on AI lately as one of the hosts (Rory Stewart) has got more concerned. Most of the c

How are people using their Max subscriptions of models? For me already the sense of 'obligation' to use a powerful model (even free offering) for all you can is a bit overwhelming and there's already a lot of work where you should just step back and think of next features yourself instead of suddenly getting spammed with a massive flawed codebase and/or even engaging in like, self-wireheading. I can only imagine others are maybe ambitiously coding a lot of features in a document or constantly asking for what next best features or making a lot of scripts or something (where they are deliverable and immediately profitable). Complaints of maxing out a '5x' subscription is definitely in another world to me at the moment. Heavily also depends on recreational/profitable uses too

Has there been any discussion here of the "leaked Fable chain of thought"? https://old.reddit.com/r/ClaudeAI/comments/1ul1396/fable_5_leaked_chainofthought_in_web_interface/ In particular, does anyone have any view on if this is just a random meltdown or if this is actually what Fable's CoT looks like? If the latter, perhaps that should substantially accelerate our timelines? (On the logic that reinforcement learning can accomplish more than was understood.)