Quick Takes

Around 8 years ago Dario Amodei, Jan Leike and a few others predicted that we’d probably have AGI by now. Since then, AI capabilities have progressed so fast that many people updated that the short timelines advocates were right. And yes, their expectations about the speed of AI progress were far more directionally correct than almost anyone expected. But we don’t in fact have AGI (even under relatively weak definitions, like a reliable drop-in replacement for central examples of white-collar jobs), and the world would look very different if we did. Now a large proportion of the AI safety community is implicitly or explicitly orienting to futures where an intelligence explosion occurs within a few years. My default expectation (absent an extensive pause) is that a similar thing will happen: they'll turn out to be directionally correct (relative to the expectations of almost anyone not linked to the community) but factually wrong. Specifically, we won’t have superintelligence within the next 8 years, but things will still be moving so fast that it’ll *feel* like the people who argued for short timelines were right. I’ll write about this more extensively soon, but I wanted to say something now because it feels like the level of bandwagoning towards “singularity soon” is getting pretty wild. I previously noted that we’re seeing a pretty sharp divergence between the measured capabilities of models and their real-world impacts, which should suggest that something weird is going on. My sense is that most AI safety people are brushing this aside with the idea that automating ML research is all you need for superintelligence, without thinking very much about where and how the automated ML research would actually lead to atoms moving around in the physical world. I worry about the next decade feeling like a Shepard tone: an auditory illusion of a sound that's continually going up. In such a scenario, people would continually be panicking and grasping for big levers, witho

Some quick observations on the attacks on METR, EA, and other AI safety orgs (from my perspective as a political campaigner and former Communications Director for a union and mayor during COVID): 1. If you're explaining, you're losing: Every rebuttal repeats the accusation to people who haven't heard it yet. Answer the accusation once, link to it, and stop explaining. This is the "illusory truth" effect (information is true when it's repeated). 2. Silence doesn't work either: Not responding on the record suggests something is being hidden. Make one clear statement rather than amplifying in multiple twitter threads. 3. Concede what you can: A concept I know from political comms is the "admission against interest". This buys trust, disarms cynical voters, and establishes credibility. Saying "yes, AND here's what we're doing about it" will take some air out of the critique. 4. Messenger > message: Ideally people who know the organization being attacked, are from outside the AI safety world, have independent standing, or better yet, have disagreed with the organization publicly will land the defense better. 5. Going after people's personal lives or spouses is overreach: Ordinary people will obviously find this unfair and it's where their attacks will backfire. 6. Don't quote tweet and rebut: You're just distributing their message for them. I want to write something longer on this but don't have time right now. Lots of opportunity for good crisis comms here. Let me know what's wrong about this take and if you'd like to hear more!

Is there anyone who helps operate an AI safety / AI existential derisking talent pipeline, and who might be interested in adding a small pilot human intelligence amplification track to that pipeline? Do reach out to me and/or to @kman. (We may write a more comprehensive proposal, e.g. outlining programs of research for adult HIA (and separately for reprogenetic HIA, which may be less feasible due to perceived controversialness) that participants could work on, and explaining how we might go about mentorship. But that might come months from now, so I just wanted to float the idea if anyone happens to see this and may want to run with it.)

Palisade is looking to hire experienced video editors, on a contract basis, to start immediately. Apply here. Palisade is working on an urgent, high-leverage, video project directly addressing what we believe to be a key bottleneck to the world’s ability to orient to AI risk[1]. We want to ship that project ASAP and we’re currently bottlenecked on editing capacity.  If you are an experienced video editor that could show up in our office in Berkeley in the next three days, and quickly get up to speed with our editing pipeline, we would like to hire you at $100 an hour for approximately the next 30 days. Desiderata:  * You must be experienced enough to be thrown into a fast and chaotic situation and do a good job, while recieving direction, but minimal hand-holding. * You must be able to work in person in Berkeley. * You have a workstation (laptop, desktop, whatever) that you can bring * Context on the AI risk situation is a plus. * Ability to work (some) weekend days is a plus. If you are interested, apply here. If you seem like a good fit, we’ll reach out and schedule a quick video interview. From there,  we’ll start with a 2-3 day work trial. If that work trial goes well, we expect to want to keep you on until this project (and possibly several other projects that are in the works) is completed.  This is a contract role, but if you execute well, there’s a good chance we would want to hire you long term as well. If you know experienced video editors, send them this post! We’ll pay you a $1000 referral bonus for every person we end up hiring! 1. ^ We're not ready to announce it publicly, but I, personally, think it's pretty awesome. I currently expect that this will turn out to be the single project, out of everything that I've done in my career to date, that I am most proud of.

Data point: I attended the London Haskell meetup today and told a few people I was on sabbatical and hoping to help with AI safety, because it's clearly the most important thing right now. Reactions ranged from neutral to "oh yeah, obviously"; no one registered to me as surprised or skeptical.

People with p(doom) > 50%: what concrete empirical achievement on current or near-future models, if any, would bring your p(doom) under 25%? Answers could be anything from "the steering vector for corrigibility generalizes surprisingly far" to "we completely reverse-engineer GPT-4 GPT-6 and build a trillion-parameter GOFAI without any deep learning". (I last asked this question two years ago)

I keep running into (understandably) confused takes on "If the CEOs of these companies really believe AI might kill everyone... why would they start a company to build AI?" and rather than try to explain with nuance in words: Suggested refinements are welcome.