Not today, but they will soon. They're working toward controlling all releases, and then they will start working on controlling the research itself, though they don't know yet that this is what they will soon want to do.
Nationalization is a step toward a pause. If the only entities at the table are states, the coordination problem becomes a lot easier.
No, but it's a great sign! It wouldn't be sane for any government to be friendly with the AI companies, so I smile when they fight. A stable multilateral pause is downstream from the vibes as well, not only from good policy.
1It's not a crux in my view. I think the choice of method (LLM/modified-from-LLM/non-LLM) has some impact on the timeline distribution, but not on policy or alignment research.
Trying to make an intervention narrow enough that it only covers one architecture or research area probably won't succeed to cover even that much, and we probably need to worry about multiple risk vectors.
I believe the kind of alignment research that is based on LLMs is net-negative, since it safety-washes a fundamentally dangerous and morally bankrupt pursuit of superintelligence. We...
1I strongly agree with your perspective on what high integrity and honesty entail!
I have been in the PauseAI movement since August 2023, and I cannot recall ever observing PauseAI Global or PauseAI US as organizations behaving recklessly in regard to the truth. We inherited a lot of rationalist obsession with honesty, which has predominantly been a very good thing. Holly's alleged recklessness in point of fact does not bleed over to PauseAI US's communications with the public or policymakers.
Holly as an individual has personal beef with many people in the r...
2That is very strange. I associate the strategy of being upfront, honest, precise, and focused about AI extinction risk very strongly with ControlAI, second only to MIRI.
ControlAI's first campaign was about deepfakes. Maybe the only significantly indirect thing they've ever done, and it still wasn't what I would call deceptive. Ever since then, they have been focused on extinction risk, which can be clearly seen from their campaign statements signed by politicians. Their social media campaigns have all kinds of hooks about possible negative AI futures, but ...
We are much more in agreement than I expected! Thank you for clarifying. I agree with every point you made in this comment. (Excepting the comparison to continental philosophy, which is new to me and not something I think I can evaluate one way or the other.)
1The shape of superintelligence that we are building isn't necessarily the shape of superintelligence that will ultimately exist, though, right? That's one of the hard barriers to see beyond: the ability for an AI to invent new AI paradigms.
It's better than useless to study current systems if they give us insight into intelligence in general, and we might get useful information about the types of minds that are likely to exist. (I find work like the Natural Abstractions Hypothesis really interesting, and that benefits from empirical work). But that isn't quite the same as studying things that are shaped like the superintelligence we will get.
I think what you perceive as an aversion to empiricism and rigorous justification is just an acknowledgment of the areas where empiricism and rigorous justification are currently impossible.
It is very common for AI safety researchers to fall for the trap of doing mostly useless (or counterproductive) empirical work rather than trying to solve important problems. We know that something abstract like agent foundations will be necessary, because it is strictly impossible to do empirical work on superintelligence if superintelligence doesn't exist. (You probab...
1
1
1With a very short, messy pause, I expect humanity to be long gone before 2040. I am much more interested in the economic outlook of a decades-long or permanent moratorium, since that's the only version of the future we should expect to observe.
That the world is highly engineerable, which can lead to the relaxation or abolition of seemingly hard bottlenecks. Also that the world can respond extremely quickly to implement those changes when the incentives are right.
Overpopulation would have been a massive problem at different points in history if not for the invention of horseless transport and high-yield, resilient cereal crops. People living in New York City in the late 1800's or in developing nations in the 1960s and 70s were rescued from the worst hazards of overpopulation because of the motorc...
Here's the Metaculus data. If you click "Show Projection" on the graph, you'll see the trendline crossing "Metaculus Pro Forecasters" (which includes myself) in June 2027. That closely matches the current projection of ForecastBench, which puts the estimated LLM-superforecaster parity in May 2027. The closeness of the match may be coincidental to some degree, since the two running experiments use very different methods and targets, but I find the general agreement between these unrelated projects pretty compelling.
I asked the free versions of ChatGPT, Clau...
I don't understand the forecasting community's love of markets. Metaculus does just fine without a market system, and no one is destroying their lives using it.
Of course, now that the machine forecasting method has started, it will not take long to outstrip our feeble powers, so to speak. On the current trend, AI will be strictly superhuman at forecasting by mid next year, and the term "forecaster" may go the way of "computer," as a description of a machine rather than a human pursuit.
In that case, markets are again unnecessary overhead, compared to e.g. just asking 5 AI systems a couple times and taking the median prediction.
What kind of plays are there now, other than short-term?
I would really rather things come together long before the final hour, if we can at all help it. I would like to see that navigation take place over decades, if reality permits such sanity.
And while I do like imagining it, most people have failed to rise to the occasion so far, especially those in the AI companies. It's up to the general public now, to wake up the governments of the world and shut down the race.
My gut-check is to imagine a superintelligence that does the complicated acausal multiversal thing, and then separately imagine a superintelligence that just tries to take over the lightcone with no hesitation. Whichever ends up winning more stuff is the real one and has the right decision theory.
I just can't imagine a superintelligence being trolled by too-clever-by-half decision theory arguments, when it can just kill you and take your stuff (and the whole empty galaxy) instead. Maybe it has to modify the intergalactic probes so that it's provably the ki...
That sounds appealing if almost nothing ever changes, but we should expect discontinuities due to each or any of:
We just don't understand any of this stuff. Even a perfec...
The mechanism isn't [public disapproval] -> [AI moratorium]. That sounds hopeless to me, too. What I'm actually relatively optimistic about is [public awareness] -> [educating policymakers] + [signal to policymakers that they are allowed to do something about it] -> [common knowledge of the desire to act] -> [AI moratorium]. This feels very doable to me after getting back from speaking with my congressional offices in DC.
The key here is that most politicians don't want to take these risks either. Really very few people are on board with the who...
You can pursue both goals, but you have chosen not to, and I find that alarming.
There are many, many concrete accusations in this post. It doesn't particularly matter whether there is a "main" one. Vassar is a serial psychological and sexual abuser, and has caused multiple deaths. That is bad.
Plex opening up about his experiences with this abuser was a terrifying thing to do. The appropriate response in this circumstance is to respond with empathy. If you aren't going to do that, it is better not to say anything at all. I understand that we are on LessWron...
The government is made of many people, and both reasons are at play. If a more cyber- and bio-capable AI is broadly released by another company and the White House does nothing to impede them, I will be very surprised.