"A fight between ‘Big Tech’ and ‘Silicon Valley’..."
I'm mystified. What are 'Big Tech' and 'Silicon Valley' supposed to refer to? My guess would have been that they are synonyms, but apparently not...
Roughly this, yes. SV here means the startup ecosystem, Big Tech means large established (presumably public) companies.
From https://en.wikipedia.org/wiki/Santa_Clara%2C_California
"Santa Clara is located in the center of Silicon Valley and is home to the headquarters of companies such as Intel, Advanced Micro Devices, and Nvidia."
So I think you shouldn't try to convey the idea of "startup" with the metonym "Silicon Valley". More generally, I'd guess that you don't really want to write for a tiny audience of people whose cultural references exactly match your own.
Tyler Cowen’s rather bold claim that May 2024 will be remembered as the month that the AI safety movement died.
What really seems to have died, is the idea of achieving AI safety, via the self-restraint of AI companies. Instead, they will rely on governments and regulators to restrain them.
It was always going to come down to the strong arm of the law to beat AI companies into submission. I was always under the impression that attempts at alignment or internal company restraints were hypothetical thought experiments (no offence). This has been the reality of the world with all inventions, not just AI.
Unfortunately, both sides (lawyers & researchers) seem unwilling to find a middle-ground which accommodates the strengths of each and mitigates the misunderstandings in both camps.
Feeling pessimistic after reading this.
In terms of things that go in AI updates, this has been the busiest two week period so far. Every day ends with more open tabs than it started, even within AI.
As a result, some important topics are getting pushed to whenever I can give them proper attention. Triage is the watchword.
In particular, this post will NOT attempt to cover:
My guess is at least six of these eight get their own posts (everything but #3 and #8).
So here is the middle third: The topics I can cover here, and are still making the cut.
Still has a lot of important stuff in there.
Table of Contents
From this week: Do Not Mess With Scarlett Johansson, On Dwarkesh’s Podcast with OpenAI’s John Schulman, OpenAI: Exodus, GPT-4o My and Google I/O Day
Language Models Offer Mundane Utility
If at first you don’t succeed, try try again. For Gemini in particular, ‘repeat the question exactly in the same thread’ has had a very good hit rate for me on resolving false refusals.
Claim that GPT-4o gets greatly improved performance on text documents if you put them in Latex format, vastly improving effective context window size.
Rowan Cheung strongly endorses the Zapier Central Chrome extension as an AI tool.
Get a summary of the feedback from your practice demo on Zoom.
Get inflation expectations, and see how they vary based on your information sources. Paper does not seem to focus on the questions I would find most interesting here.
Sully is here for some of your benchmark needs.
Find out if you are the asshole.
Why not both? I predict both. If AIs are recording and analyzing everything we do, then people will obviously start optimizing their choices to get the results they want from the AIs. I would not presume this will mean that a ‘be shallower’ strategy is the way to go, for example LLMs are great and sensing the vibe that you’re being shallow, and also their analysis should get less shallow over time and larger context windows. But yeah, obviously this is one of those paths that leads to the dark side.
Ask for a one paragraph Strassian summary. Number four will not shock you.
Own your HOA and its unsubstantiated violations, by taking their dump of all their records that they tried to overwhelm you with, using a script to convert to text, using OpenAI to get the data into JSON and putting it into a Google map, proving the selective enforcement. Total API cost: $9. Then they found the culprit and set a trap.
Get greatly enriched NBA game data and estimate shot chances. This is very cool, and even in this early state seems like it would enhance my enjoyment of watching or the ability of a team to do well. The harder and most valuable parts still lay ahead.
Turn all your unstructured business data into what is effectively structured business data, because you can run AI queries on it. Aaron Levie says this is why he is incredibly bullish on AI. I see this as right in the sense that this alone should make you bullish, and wrong in the sense that this is far from the central thing happening.
Or someone else’s data, too. Matt Bruenig levels up, uses Gemini Flash to extract all the NLRB case data, then uses ChatGPT to get a Python script to turn it into clickable summaries. 66k cases, output looks like this.
Language Models Don’t Offer Mundane Utility
Would you like some ads with that? Link has a video highlighting some of the ads.
Seriously, Google, if I want to use Gemini (and often I do) I will use Gemini.
The good news is they have gotten a bit better about this. I did a check after I saw this, and suddenly there is a logic behind whether the AI answer appears. If I ask for something straightforward, I get a normal result. If I ask for something using English grammar, and imply I have something more complex, then the AI comes out. That’s not an entirely unreasonable default.
The other good news is there is a broader fix. Ernie Smith reports that if you add “udm=14” to the end of your Google search, this defaults you into the new Web mode. If this is for you, GPT-4o suggests using Tampermonkey to append this automatically, or you can use this page on Chrome to set defaults.
American harmlessness versus Chinese harmlessness. Or, rather, American helpfulness versus Chinese unhelpfulness. The ‘first line treatment’ for psychosis is not ‘choose from this list of medications’ it is ‘get thee to a doctor.’ GPT-4o gets an A on both questions, DeepSeek-V2 gets a generous C maybe for the first one and an incomplete on the second one. This is who we are worried about?
OpenAI versus Google
What kind of competition is this?
Whereas here’s my view on that.
As in, they are two companies trying very hard to be cool and hip, in a way that makes it very obvious that this is what they are doing. Who is ‘right’ versus ‘wrong’? I have no idea. It is plausible both were ‘right’ given their goals and limitations. It is also plausible that this is part of Google being horribly bad at presentations. Perhaps next time they should ask Gemini for help.
I do think ‘OpenAI won’ the presentation war, in the sense that they got the hype and talk they wanted, and as far as I can tell Google got a lot less, far in excess of the magnitude of any difference in the underlying announcements and offerings. Well played, OpenAI. But I don’t think this is because of the background of their set.
I also think that if this is what sticks in Altman’s mind, and illustrates where his head is at, that could help explain some other events from the past week.
I would not go as far as Teortaxes here, but directionally they have a point.
GPT-4o My
Ethan Mollick reports on why GPT-4o matters. He thinks, highly plausibly, that the biggest deal is free access. He does not mention the speed boost or API price drop, and is looking forward to trying the multimodal features but lacks access. He emphasizes the shift from ‘make the model smarter’ to adding features that enhance mundane utility.
Alexandr Wang thinks the big emphasis is that post-training is getting more important. In his mind, GPT-4o now rules the roost for large models, Gemini 1.5 Flash at only 1M now rules for very small ones. And he notes that both companies have versions of many of the same offerings, coming online at similar times.
I am suspicious. I am especially suspicious, as I continue to use it, that GPT-4o in text only mode is not so great aside from its speed, that it made other sacrifices (such as probably size) to get that speed and that it ‘wins in the Arena’ because the post-training aimed at winning in the Arena. That still gives it a potentially big edge once the multi-modal abilities come properly online, and I’m excited to see where that goes.
One way you can make people prefer your model’s outputs is to give the user what they request when another model would have refused?
That is a rather dramatic chart. In terms of the direct consequences of users entering queries, I am fine with GPT-4o being easily jailbroken. You can still jailbreak Claude Opus if you care enough and there’s nothing that dangerous to be done once you do.
I still look to such questions as canaries in the coal mine. The first job of your safety department is to get the models that exist today to not do, today, the things you have explicitly decided you do not want your models to do. Ideally that would be a fully robust regime where no one can jailbreak you, but I for now will settle for ‘we decided on purpose to made this a reasonable amount of hard to do, and we succeeded.’
If OpenAI had announced something like ‘after watching GPT-4-level models for a year, we have decided that robust jailbreak protections degrade performance while not providing much safety, so we scaled back our efforts on purpose’ then I do not love that, and I worry about that philosophy and your current lack of ability to do safety efficiently at all, but as a deployment decision, okay, fine. I have not heard such a statement.
There are definitely a decent number of people who think GPT-4o is a step down from GPT-4-Turbo in the ways they care about.
The phantom pattern matching is impossible to miss, and a cause of many of the stupidest mistakes.
The GPT-4o trademark, only entered (allegedly) on May 16, 2024 (direct link).
Claim that the link contains the GPT-4o system prompt. There is nothing here that is surprising given prior system prompts. If you want GPT-4o to use its browsing ability, best way is to tell it directly to do so, either in general or by providing sources.
Responsable Scaling Policies
Anthropic offers reflections on their responsible scaling policy.
They note that with things changing so quickly they do not wish to make binding commitments lightly. I get that. The solution is presumably to word the commitments carefully, to allow for the right forms of modification.
Here is how they summarize their actual commitments:
One issue is that experts disagree on which potential capabilities are dangerous, and it is difficult to know what future abilities will manifest, and all testing methods have their flaws.
Then we get to their central focus, which has been on setting their ASL-3 standard. What would be sufficient defenses and mitigations for a model where even a low rate of misuse could be catastrophic?
For human misuse they expect a defense-in-depth approach, using a combination of RLHF, CAI, classifiers of misuse at multiple stages, incident reports and jailbreak patching. And they intend to red team extensively.
This makes me sigh and frown. I am not saying it could never work. I am however saying that there is no record of anyone making such a system work, and if it would work later it seems like it should be workable now?
Whereas all the major LLMs, including Claude Opus, currently have well-known, fully effective and fully unpatched jailbreaks, that allow the user to do anything they want.
An obvious proposal, if this is the plan, is to ask us to pick one particular behavior that Claude Opus should never, ever do, which is not vulnerable to a pure logical filter like a regular expression. Then let’s have a prediction market in how long it takes to break that, run a prize competition, and repeat a few times.
For assurance structures they mention the excellent idea of their Impossible Mission Force (they continue to call this the ‘Alignment Stress-Testing Team’) as a second line of defense, and ensuring strong executive support and widespread distribution of reports.
My summary would be that most of this is good on the margin, although I wish they had a superior ASL-3 plan to defense in depth using currently failing techniques that I do not expect to scale well. Hopefully good testing will mean that they realize that plan is bad once they try it, if it comes to that, or even better I hope to be wrong.
The main criticisms I discussed previously are mostly unchanged for now. There is much talk of working to pay down the definitional and preparatory debts that Anthropic admits that it owes, which is great to hear. I do not yet see payments. I also do not see any changes to address criticisms of the original policy.
And they need to get moving. ASL-3 by EOY is trading at 25%, and Anthropic’s own CISO says 50% within 9 months.
There is quite a lot to do before ASL-3 is something that can be handled under the existing RSP. ASL-4 is not yet defined. ASL-3 protocols have not been identified let alone implemented. Even if the ASL-3 protocol is what they here sadly hint it is going to be, and is essentially ‘more cybersecurity and other defenses in depth and cross our fingers,’ You Are Not Ready.
Then there’s ASL-4, where if the plan is ‘the same thing only more of it’ I am terrified.
Overall, though, I want to emphasize positive reinforcement for keeping us informed.
Copyright Confrontation
Music and general training departments, not the Scarlett Johansson department.
The central demands here are explicit permission to use songs as training data, and a full explanation within a month of all ways Sony’s songs have been used.
Thread claiming many articles in support of generative AI in its struggle against copyright law and human creatives are written by lawyers and paid for by AI companies. Shocked, shocked, gambling in this establishment, all that jazz.
Deepfaketown and Botpocalypse Soon
Noah Smith writes The death (again) of the Internet as we know it. He tells a story in five parts.
I am mostly with him on the first three, and even more strongly in favor of the need to curate one’s feeds. I do think algorithmic feeds could be positive with new AI capabilities, but only if you have and use tools that customize that experience, both generally and in the moment. The problem is that most people will never (or rarely) use those tools even if offered. Rarely are they even offered.
Where on Twitter are the ‘more of this’ and ‘less of this’ buttons, in any form, that aren’t public actions? Where is your ability to tell Grok what you want to see? Yep.
For the Chinese and Russian efforts, aside from TikTok’s algorithm I think this is greatly exaggerated. Noah says it is constantly in his feeds and replies but I almost never see it and when I do it is background noise that I block on sight.
For AI, the question continues to be what we can do in response, presumably a combination of trusted sources and whitelisting plus AI for detection and filtering. From what we have seen so far, I continue to be optimistic that technical solutions will be viable for some time, to the extent that the slop is actually undesired. The question is, will some combination of platforms and users implement the solutions?
They Took Our Jobs
Avital Balwit of Anthropic writes about what is potentially [Her] Last Five Years of Work. Her predictions are actually measured, saying that knowledge work in particular looks to be largely automated soon, but she expects physical work including childcare to take far longer. So this is not a short timelines model. It is a ‘AI could automate all knowledge work while the world still looks normal but with a lot more involuntary unemployment’ model.
That seems like a highly implausible world to me. If you can automate all knowledge work, you can presumably also automate figuring out how to automate the plumber. Whereas if you cannot do this, then there should be enough tasks out there and enough additional wealth to stimulate demand that those who still want gainful employment should be able to find it. I would expect the technological optimist perspective to carry the day within that zone.
Most of her post asks about the psychological impact of this future world. She asks good questions such as: What will happen to the unemployed in her scenario? How would people fill their time? Would unemployment be mostly fine for people’s mental health if it wasn’t connected to shame? Is too much ‘free time’ bad for people, and does this effect go away if the time is spent socially?
The proposed world has contradictions in it that make it hard for me to model what happens, but my basic answer is that the humans would find various physical work and and status games and social interactions (including ‘social’ work where you play various roles for others, and also raising a family) and experiential options and educational opportunities and so on to keep people engaged if they want that. There would however be a substantial number of people who by default fall into inactivity and despair, and we’d need to help with that quite a lot.
Mostly for fun I created a Manifold Market on whether she will work in 2030.
Get Involved
Ian Hogarth gives his one-year report as Chair of the UK AI Safety Institute. They now have a team of over 30 people and are conducting pre-deployment testing, and continue to have open rolls. This is their latest interim report. Their AI agent scaffolding puts them in third place (if you combine the MMAC entries) in the GAIA leaderboard for such things. Good stuff.
They are also offering fast grants for systemic AI safety. Expectation is 20 exploratory or proof-of-concept grants with follow-ups. Must be based in the UK.
Geoffrey Irving also makes a strong case that working at AISI would be an impactful thing to do in a positive direction, and links to the careers page.
Introducing
Anthropic gives Claude tool use, via public beta in the API. It looks straightforward enough, you specify the available tools, Claude evaluates whether to use the tools available, and you can force it to if you want that. I don’t see any safeguards, so proceed accordingly.
Google Maps how has AI features, you can talk to it, or have it pull up reviews in street mode or take an immersive view of a location or search a location’s photos or the photos of the entire area around you for an item.
In my earlier experiments, Google Maps integration into Gemini was a promising feature that worked great when it worked, but it was extremely error prone and frustrating to use, to the point I gave up. Presumably this will improve over time.
Reddit and Weep
OpenAI partners with Reddit. Reddit posts, including recent ones, will become available to ChatGPT and other products. Presumably this will mean ChatGPT will be allowed to quote Reddit posts? In exchange, OpenAI will buy advertising and offer Reddit.com various AI website features.
For OpenAI, as long as the price was reasonable this seems like a big win.
It looks like a good deal for Reddit based on the market’s reaction. I would presume the key risks to Reddit are whether the user base responds in hostile fashion, and potentially having sold out cheap.
Or they may be missing an opportunity to do something even better. Yishan provides a vision of the future in this thread.
In Other AI News
OpenAI has also signed a deal with Newscorp for access to their content, which gives them the Wall Street Journal and many others.
A source tells me that OpenAI informed its employees that they will indeed update their documents regarding employee exit and vested equity. The message says no vested equity has ever actually been confiscated for failure to sign documents and it never will be.
On Monday I set up this post:
At 168 likes, we now have one employee from DeepMind, and one from Anthropic.
Jimmy Apples claimed without citing any evidence that Meta will not open source (release the weights, really) of Llama-3 405B, attributing this to a mix of SB 1047 and Dustin Moskovitz. I was unable to locate an independent source or a further explanation. He and someone reacting to him asked Yann LeCunn point blank, Yann replied with ‘Patience my blue friend. It’s still being tuned.’ For now, the Manifold market I found is not reacting continues to trade at 86% for release, so I am going to assume this was another disingenuous inception attempt to attack SB 1047 and EA.
ASML and TSMC have a kill switch for their chip manufacturing machines, for use if China invades Taiwan. Very good to hear, I’ve raised this concern privately. I would in theory love to also have ‘put the factory on a ship in an emergency and move it’ technology, but that is asking a lot. It is also very good that China knows this switch exists. It also raises the possibility of a remote kill switch for the AI chips themselves.
Did you know Nvidia beat earnings again yesterday? I notice that we are about three earnings days into ‘I assume Nvidia is going to beat earnings but I am sufficiently invested already due to appreciation so no reason to do anything more about it.’ They produce otherwise mind boggling numbers and I am Jack’s utter lack of surprise. They are slated to open above 1,000 and are doing a 10:1 forward stock split on June 7.
Toby Ord goes into questions about the Turing Test paper from last week, emphasizing that by the original definition this was impressive progress but still a failure, as humans were judged human substantially more often than all AIs. He encourages AI companies to include the original Turing Test in their model testing, which seems like a good idea.
OpenAI has a super cool old-fashioned library. Cade Metz here tries to suggest what each book selection from OpenAI’s staff might mean, saying more about how he thinks than about OpenAI. I took away that they have a cool library with a wide variety of cool and awesome books.
JP Morgan says every new hire will get training in prompt engineering.
Scale.ai raises $1 billion at a $13.8 billion valuation in a ‘Series F.’ I did not know you did a Series F and if I got that far I would skip to a G, but hey.
Suno.ai Raises $125 million for music generation.
New dataset from Epoch AI attempting to hart every model trained with over 10^23 flops (direct). Missing Claude Opus, presumably because we don’t know the number.
Not necessarily the news department: OpenAI publishes a ten-point safety update. The biggest update is that none of this has anything to do with superalignment, or with the safety or alignment of future models. This is all current mundane safety, plus a promise to abide by the preparedness framework requirements. There is a lot of patting themselves on the back for how safe everything is, and no new initiatives, although this was never intended to be that sort of document.
Then finally there’s this:
Hahahahahahahahahahahahahahahahahahaha.
That does not mean that mundane safety concerns are a small thing.
I Spy With My AI (or Total Recall)
Why let the AI out of the box when you can put the entire box into the AI?
Microsoft also announced live caption translations, auto super resolution upscaling on apps (yes with a toggle for each app, wait those are programs, wtf), AI in paint and automatic blurring (do not want).
This is all part of the new ‘Copilot+’ offering for select new PCs, including their new Microsoft Surface machines. You will need a Snapdragon X Elite and X Plus, 40 TOPs, 225 GB of storage and 16 GB RAM. Intel and AMD chips can’t cut it (yet) but they are working on that.
(Consumer feedback report: I have a Microsoft Surface from a few years ago, it was not worth the price and the charger is so finicky it makes me want to throw things. Would not buy again.)
I would hope this would at least be opt-in. Kevin Beaumont reports it will be opt-out, citing this web page from Microsoft. It appears to be enabled by default on Copilot+ computers. My lord.
At minimum, even if you do turn it off, it does not seem that hard to turn back on:
I would also not trust a Windows update to not silently turn it back on.
The UK Information Commissioner’s Office (ICO) is looking into this, because yeah.
In case it was not obvious, you should either:
I am not here to tell you which of those is the play.
I only claim that it seems that soon you must choose.
If the feature is useful, a large number of people are going to choose option one.
I presume almost no one will pick option two, except perhaps for gaming PCs.
Option three is viable.
If there is one thing we have learned during the rise of AI, and indeed during the rise of computers and the internet, it is that almost all people will sign away their privacy and technological vulnerability for a little mundane utility, such as easier access to cute pictures of cats.
Yelling at them that they are being complete idiots is a known ineffective response.
And who is to say they even are being idiots? Security through obscurity is, for many people, a viable strategy up to a point.
Also, I predict your phone is going to do a version of this for you by default within a few years, once the compute and other resources are available for it. I created a market on how quickly. Microsoft is going out on far less of a limb than it might look like.
In any case, how much mundane utility is available?
Quite a bit. You would essentially be able to remember everything, ask the AI about everything, have it take care of increasingly complex tasks with full context, and this will improve steadily over time, and it will customize to what you care about.
If you ignore all the obvious horrendous downsides of giving an AI this level of access to your computer, and the AI behind it is good, this is very clearly The Way.
There are of course some people who will not do this.
How long before they are under increasing pressure to do it? How long until it becomes highly suspicious, as if they have something to hide? How long until it becomes a legal requirement, at best in certain industries like finance?
Ben Thompson, on the other hand, was impressed, calling the announcement event ‘the physical manifestation of CEO Satya Nadella’s greatest triumph’ and ‘one of the most compelling events I’ve attended in a long time.’ Ben did not mention the privacy and security issues.
Quiet Speculations
Ethan Mollick perspective on model improvements and potential AGI. He warns that AIs are more like aliens that get good at tasks one by one, and when they are good they by default get very good at that task quickly, but they are good at different things than we are, and over time that list expands. I wonder to what extent this is real versus the extent this is inevitable when using human performance as a benchmark while capabilities steadily improve, so long as machines have comparative advantages and disadvantages. If the trends continue, then it sure seems like the set of things they are better at trends towards everything.
Arthur Breitman suggests Apple isn’t developing LLMs because there is enough competition that they are not worried about vender lock-in, and distribution matters more. Why produce an internal sub-par product? This might be wise.
Microsoft CTO Kevin Scott claims ‘we are nowhere near the point of diminishing marginal returns on how powerful we can make AI models as we increase the scale of compute.’
Gary Marcus offered to Kevin Scott him $100k on that.
This was a truly weird speech on future challenges of AI by Randall Kroszner, external member of the Financial Policy Committee of the Bank of England. He talks about misalignment and interpretability, somehow. Kind of. He cites the Goldman Sacks estimate of 1.5% labor productivity and 7% GDP growth over 10 years following widespread AI adaptation, that somehow people say with a straight face, then the flip side is McKinsey saying 0.6% annual labor productivity growth by 2040, which is also not something I could say with a straight face. And he talks about disruptions and innovation aids and productivity estimation J-curves. It all sounds so… normal? Except with a bunch of things spiking through. I kept having to stop to just say to myself ‘my lord that is so weird.’
Politico is at it Again
Politico is at it again. Once again, the framing is a background assumption that any safety concerns or fears in Washington are fake, and the coming regulatory war is a combination of two fights over Lenin’s question of who benefits.
That’s it. Those are the issues and stakes in play. Nothing else.
How dismissive is this of safety? Here are the two times ‘safety’ is mentioned:
Testing standards are ‘easy things to find agreement on’? Fact check: Lol, lmao.
That’s it. The word ‘risk’ appears twice and neither has anything to do with safety. Other words like ‘capability,’ ‘existential’ or any form of ‘catastrophic’ do not appear. It is all treated as obviously irrelevant.
The progress is here they stopped trying to bulk up people worried about safety as boogeymen (perhaps because this is written by Matthew Kaminski, not Brendon Bordelon), and instead point to actual corporations that are indeed pursuing actual profits, with Silicon Valley taking on Big Tech. And I very much appreciate that ‘open source advocates’ has now been properly identified as Silicon Valley pursuing its business interests.
Notice the escalation. This is not ‘Big Tech wants regulatory capture to actively enshrine its advantages, and safety is a Big Tech plot.’ This is ‘Silicon Valley wants to actively use regulatory action to prevent Big Tech from winning,’ with warnings that attempts to not have a proper arms race to ever more capable systems will cause intervention from regulators. By ‘more open market’ they mean ‘government intervention in the market,’ government’s favorite kind of new freer market.
As I have said previously, we desperately need to ensure that there are targeted antitrust exemptions available so that when AI labs can legally collaborate around safety issues they are not accused of collusion. It would be completely insane to not do this.
And as I keep saying, open source advocates are not asking for a level playing field or a lack of government oppression. They are asking for special treatment, to be exempt from the rules of society and the consequences of their actions, and also for the government to directly cripple their opponents for them.
Are they against regulatory capture? Only if they don’t get to do the capturing.
Then there is the second track, the question of guardrails that might spoil the ‘libertarian sandbox,’ which neither ‘side’ of tech wants here.
Here is the two mentions of ‘risk’:
I once again have been roped into extensively covering a Politico article, because it is genuinely a different form of inception than the previous Politico inception attempts. But let us continue to update that Politico is extraordinarily disingenuous and hostilely motivated on the subject of AI regulation. This is de facto enemy action.
Here, Shakeel points out the obvious central point being made here, which is that most of the money and power in this fight is Big Tech companies fighting not only to avoid any regulations at all, but to get exemptions from other ordinary rules of society. When ethics advocates portray notkilleveryoneism (or safety) advocates as their opponents, that is their refusal to work together towards common goals and also it misses the point. Similarly, here Seán Ó hÉigeartaigh expresses concern about divide-and-conquer tactics targeting these two groups despite frequently overlapping and usually at least complementary proposals and goals.
Or perhaps the idea is to illustrate that all the major players in Tech are aligned in being motivated by profit and in dismissing all safety concerns as fake? And a warning that Washington is in danger of being convinced? I would love that to be true. I do not think a place like Politico works that subtle these days, nor do I expect those who need to hear that message to figure out that it is there.
Beating China
If we care about beating China, by far the most valuable thing we can do is allow more high-skilled immigration. Many of their best and brightest want to become Americans.
This is true across the board, for all aspects of our great power competition.
It also applies to AI.
From his thread about the Schumer report:
I suspect the numbers are even more lopsided than this graph suggests.
To what extent is being in America a key element of being a top-tier AI researcher? How many of these same people would have been great if they had stayed at home? If they had stayed at home, would others have taken their place here in America? We do not know. I do know it is essentially impossible that this extent is so large we would not want to bring such people here.
Do we need to worry about those immigrants being a security risk, if they come from certain nations like China and we were to put them into OpenAI, Anthropic or DeepMind? Yes, that does seem like a problem. But there are plenty of other places they could go, where it is much less of a problem.
The Quest for Sane Regulations
Labour vows to force firms developing powerful AI to meet requirements.
Unless something very unexpected happens, they will win the next election, which is currently scheduled for July 4.
This is indeed the a16z dilemma:
SB 1047 Update
The California Senate has passed SB 1047, by a vote of 32-1.
An attempt to find an estimate of the costs of compliance with SB 1047. The attempt appears to fail, despite some good discussions.
This seems worth noting given the OpenAI situation last week:
Scott Wiener Twitter thread and full open letter on SB 1047.
This very explicitly clarifies the intent of the bill across multiple misconceptions and objections, all in line with my previous understanding.
They actively continue to solicit feedback and are considering changes.
If you are concerned about the impact of this bill, and feel it is badly designed or has flaws, the best thing you can do is offer specific critiques and proposed changes.
I strongly agree with Weiner that this bill is light touch relative to alternative options. I see Pareto improvements we could make, but I do not see any fundamentally different lighter touch proposals that accomplish what this bill sets out to do.
I will sometimes say of a safety bill, sometimes in detail: It’s a good bill, sir.
Other times, I will say: It’s a potentially good bill, sir, if they fix this issue.
That is where I am at with SB 1047. Most of the bill seems very good, an attempt to act with as light a touch as possible. There are still a few issues with it. The derivative model definition as it currently exists is the potential showstopper bug.
To summarize the issue once more: As written, if interpreted literally and as I understand it, it allows developers to define themselves as derivative of an existing model. This, again if interpreted literally, lets them evade all responsibilities, and move those onto essentially any covered open model of the same size. That means both that any unsafe actor goes unrestricted (whether they be open or closed), and releasing the weights of a covered model creates liability no matter how responsible you were, since they can effectively start the training over from scratch.
Scott Weiner says he is working on a fix. I believe the correct fix is a compute threshold for additional training, over which a model is no longer derivative, and the responsibilities under SB 1047 would then pass to the new developer or fine-tuner. Some open model advocates demand that responsibility for derivative models be removed entirely, but that would transparently defeat the purpose of preventing catastrophic harm. Who cares if your model is safe untuned, if you can fine-tune it to be unsafe in an hour with $100?
Then at other times, I will look at a safety or other regulatory bill or proposal, and say…
That’s Not a Good Idea
So it seems only fair to highlight some not good ideas, and say: Not a good idea.
One toy example would be the periodic complaints about Section 230. Here is a thread on the latest such hearing this week, pointing out what would happen without it, and the absurdity of the accusations being thrown around. Some witnesses are saying 230 is not needed to guard platforms against litigation, whereas it was created because people were suing platforms.
Adam Thierer reports there are witnesses saying the Like and Thumbs Up buttons are dangerous and should be regulated.
Brad Polumbo here claims that GLAAD says Big Tech companies ‘should cease the practice of targeted surveillance advertising, including the use of algorithmic content recommendation.’
From April 23, Adam Thierer talks about proposals to mandate ‘algorithmic audits and impact assessments,’ which he calls ‘NEPA for AI.’ Here we have Assembly Bill 2930, requiring impact assessments by developers, and charge $25,000 per instance of ‘algorithmic discrimination.’
Another example would be Colorado passing SB24-205, Consumer Protections for Artificial Intelligence, which is concerned with algorithmic bias. Governor Jared Polis signed with reservations. Dean Ball has a critique here, highlighting ambiguity in the writing, but noting they have two full years to fix that before it goes into effect.
I would be less concerned with the ambiguity, and more concerned about much of the actual intent and the various proactive requirements. I could make a strong case that some of the stuff here is kind of insane, and also seems like a generic GPDR-style ‘you have to notify everyone that AI was involved in every meaningful decision ever.’ The requirements apply regardless of size, and worry about impacts that are the kind of thing society can mitigate as we go.
The good news is that there are also some good provisions like IDing AIs, and also full enforcement of the bad parts seems impossible? I am very frustrated that a bill that isn’t trying to address catastrophic risks, but seems far harder to comply with, and seems far worse to me than SB 1047, seems to mostly get a pass. Then again, it’s only Colorado.
I do worry about Gell-Mann amnesia. I have seen so many hyperbolic statements, and outright false statements, about AI bills, often from the same people that point out what seem like obviously horrible other proposed regulatory bills and policies. How can one trust their statements about the other bills, short of reading the actual bills (RTFB)? If it turned out they were wrong, and this time the bill was actually reasonable, who would point this out?
So far, when I have dug deeper, the bills do indeed almost always turn out to be terrible, but the ‘rumors of the death of the internet’ or similar potential consequences are often greatly exaggerated. The bills are indeed reliably terrible, but not as terrible as claimed. Alas, I must repeat my lament that I know of no RTFB person I can turn to on other topics, and my cup doth overflow.
The Week in Audio
I return to the Cognitive Revolution to discuss various events of the past week first in part one, then this is part two. Recorded on Friday, things have changed by the time you read this.
From last week’s backlog: Dwarkesh Patel as guest on 80k After Hours. Not full of gold on the level of Dwarkesh interviewing others, and only partly about AI. There is definitely gold in those hills for those who want to go into these EA-related weeds. If you don’t want that then skip this one.
Around 51:45 Dwarkesh notes there is no ‘Matt Levine for AI’ and that picking up that mantle would be a good thing to do. I suppose I still have my work cut out.
A lot of talk about EA and 80k Hours ways of thinking about how to choose paths in life, that I think illustrates well both the ways it is good (actively making choices rather than sleepwalking, having priorities) and not as good (heavily favoring the legible).
Some key factors in giving career advice they point out are that from a global perspective power laws apply and the biggest impacts are a huge share of what matters, and that much advice (such as ‘don’t start a company in college’) is only good advice because the people to whom it is horribly bad advice will predictably ignore it.
Rhetorical Innovation
Why does this section exist? This is a remarkably large fraction of why.
This is a Platonic encapsulation of this class of argument:
[it continues from there, but gets less interesting and stops being Platonic.]
The latest reiteration of why p(doom) is useful even if highly imprecise, and why probabilities and probability ranges are super useful in general for communicating your actual epistemic state. In particular, that when Jan Leike puts his at ‘10%-90%’ this is a highly meaningful and useful statement of what assessments he considers reasonable given the evidence, providing much more information than saying ‘I don’t know.’ It is also more information than ‘50%.’
For the record: This, unrelated to AI, is the proper use of the word ‘doomer.’
The usual suspects, including Bengio, Hinton, Yao and 22 others, write the usual arguments in the hopes of finally getting it right, this time as Managing Extreme AI Risks Amid Rapid Progress in Science.
I rarely see statements like this, so it was noteworthy that someone noticed.
Aligning a Smarter Than Human Intelligence is Difficult
It is even more difficult when you don’t know what ‘alignment’ means.
Which, periodic reminder, you don’t.
Here we have one attempt to choose a definition, and cases for and against it:
Good advice given everything we know these days:
That does not express a strong opinion on whether we currently know of a better plan.
And it is exceedingly difficult when you do not attempt to solve the problem.
Dean Ball says here, in the most thoughtful version I have seen of this position by far, that the dissolution of the Superalignment team was good because distinct safety teams create oppositionalism, become myopic about box checking and employee policing rather than converging on the spirit of actual safety. Much better to diffuse the safety efforts throughout the various teams. Ball does note that this does not apply to the extent the team was doing basic research.
There are three reasons this viewpoint seems highly implausible to me.
Thus the biggest disagreement here, in my view, which is when he says this:
Yes, it is bigger than a technical problem, and that is important.
OpenAI has not made ‘meaningful progress.’ Certainly we are not on track to solve such problems, and we should not presume they will essentially solve themselves with an ordinary effort, as is implied here.
Indeed, with that attitude, it’s Margaritaville (as in, we might as well start drinking Margaritas.1) Whereas with the attitude of Leike and Sutskever, I disagreed with their approach, but I could have been wrong or they could have course corrected, if they had been given the resources to try.
Nor is the second phase problem that we also must solve well-described by ‘basically politics’ of a type we are used to, because there will be entities involved that are not human. Our classical liberal political solutions work better than known alternatives, and well enough for humans to flourish, by assuming various properties of humans and the affordances available to them. AIs with far greater intelligence, capabilities and efficiency, that can be freely copied, and so on, would break those assumptions.
I do greatly appreciate the self-awareness and honesty in this section:
It is notable how many AI safety advocates, when discussing almost any topic except transformational AI, are also classical liberals. If this confuses you, notice that.
The Lighter Side
Not under the current paradigm, but worth noticing.
Also, yes, it really is this easy.
And yet, somehow it is still this hard? (I was not able to replicate this one, may be fake)
It’s a fun game.
Sometimes you stick the pieces together and know where it comes from.
A problem statement:
It’s coming.
Are you ready?
1
Also, ‘some people say that there’s a woman to blame, but I know it’s my own damn fault.’