reasoning (o3) is largely solved
Solving competition problems could well be like a chess-playing AI playing chess well. Does it generalize far enough, can the method be applied to train the AI on a wide variety of tasks that are not like competition problems (distinct in it being possible to write verifiers for attempted solutions)? We know that this is not the case with AlphaZero. Is it the case with o3-like methods? Hard to tell. I don't see how it could be known either way yet.
I want to register different probabilities:
- Jobs hits first (25%).
- AGI race hits first (50%).
- Alignment hits first (15%).
The below captures some of my thoughts on the "Jobs hits first" scenario.
There is probably something with the sort of break down I have in mind (though, whatever this may be, I have not encountered it, yet) w.r.t. Jobs hits first but here goes: the landscape of AI induced unemployment seems highly heterogeneous, at least in expectation over the next 5 years. For some jobs, it seems likely that there will be instances of (1) partial automation, which could mean either not all workers are no longer needed (even if their new tasks no longer fully resemble their previous tasks), i.e. repurposing of existing workers, or most workers remain employed but do more labor with help from AI and of (2) prevalent usage of AI across the occupational landscape but without much unemployment, with human work (same roles pre-automation) still being sought after, even if there are higher relative costs associated with the human employment (basically, the situation: human work + AI work > AI work, after considering all costs). Updating based on this more nuanced (but not critically evaluated) situation w.r.t. Jobs hits first, I would not expect the demand for a halt to job automation to be any less but perhaps a protracted struggle over what jobs get automated might be less coordinated if there are swaths of the working population still holding out career-hope, on the basis that they have not had their career fully stripped away, having possibly instead been repurposed or compensated less conditional on the automation.
The phrasing
...a protracted struggle...
nevertheless seems a fitting description for the near-term future of employment as it pertains to the incorporation of AI into work.
perhaps a protracted struggle over what jobs get automated might be less coordinated if there are swaths of the working population still holding out career-hope, on the basis that they have not had their career fully stripped away, having possibly instead been repurposed or compensated less conditional on the automation.
Yeah, this is totally what I have in mind. There will be some losers and some big winners, and all of politics will be about this fact more or less. (think the dockworkers strike but 1000x)
Brain surgeon is the prototypical "goes last"example:
- a "human touch" is considered a key part of the health care
- doctors have strong regulatory protections limiting competition
- Literal lives at at stake and medical malpractice is one of the most legally perilous areas imaginable
Is neuralink the exception that proves the rule here? I imagine that IF we come up with live saving or miracle treatments that can only be done with robotic surgeons, we may find a way through the red tape?
Learning on-the-fly remains, but I expect some combination of sim2real and muZero to work here.
Hmm? sim2real AFAICT is an approach to generating synthetic data, not to learning. MuZero is a system that can learn to play a bunch of games, with an architecture very unlike LLMs. This sentence doesn't typecheck for me; what way of combining these concepts with LLMs are you imagining?
Imagine you were trying to build a robot that could:
1. Solve a complex mechanical puzzle it has never seen before
2. Play at an expert level a board game that I invented just now.
Both of these are examples of learning-on-the-fly. No amount of pre-training will ever produce a satisfying result.
The way I believe a human (or a cat) solves 1. is they: look at the puzzle, try some things, build a model of the toy in their head, try things on the model in their head, eventually solve the puzzle. There are efforts to get robots to follow the same process, but nothing I would consider "this is the obvious correct solution" quite yet.
The way to solve 2. (I think) is simply to have the LLM translate the rules of the game into a formal description and then run muZero on that.
Ideally there is some unified system that takes out the "translate into another domain and do your training there" step (which feels very anti-bitter-lesson). But I confess I haven't the slightest idea how to build such a system.
1Comparing IQ and codeforces doesn't make much sense. Please stop doing this.
Attaching IQs to LLMs makes even less sense. Except as a very loose metaphor. But please also stop doing this.
Is your disagreement specifically with the word "IQ" or with the broader point, that AI progress is continuing to make progress at a steady rate that implies things are going to happen soon-ish (2-4 years)?
If specifically with IQ, feel free to replace the word with "abstract units of machine intelligence" wherever appropriate.
If with "big things soon", care to make a prediction?
If specifically with IQ, feel free to replace the word with "abstract units of machine intelligence" wherever appropriate.
By calling it "IQ", you were (EDIT: the creator of that table was) saying that gpt4o is comparable to a 115 IQ human, etc. If you don't intend that claim, if that replacement would preserve your meaning, you shouldn't have called it IQ. (IMO that claim doesn't make sense — LLMs don't have human-like ability profiles.)
Two years ago, I noted that we had clearly entered the era of general intelligence, but that it was "too soon" to expect widespread social impacts.
In those 2 years, AI development has followed the best possible of the 3 paths I suggested (foom/GPT-4-takes-my-job/Slow Takeoff). Returns to scale seem to be delivering a steady ~15 IQ points/year and cutting edge models appear to be largely a compute-intensive project that allows (relatively) safety-conscious leading labs to explore the new frontiers while others reap the benefits with ~1 year delay.
If I had to identify 3 areas where GPT-3.5 was lacking, it would have been:
Of those three, reasoning (o3) is largely solved and we have promising approaches for world modeling (genie2). Learning on-the-fly remains, but I expect some combination of sim2real and muZero to work here.
Hence, while in 2023 I wrote
I would now write
Already, the first automated jobs are upon us: taxi driver, security guard, amazon worker. Which jobs will be automated next will be decided by a calculation that looks at:
Security guard at mall is the prototypical "goes first" example, since:
Brain surgeon is the prototypical "goes last"example:
As AI proliferates across society, we have to simultaneously solve a bunch of problems:
I suspect we have 2-4 years before one of these becomes a crisis. (And by crisis, I mean something everyone on Earth is talking about all the time, in the same sense that Covid-19 was a crisis).
The actual "tone" of the next few years could be very different depending on which of these crises hits first.
1. Jobs hits first. In this world, mass panic about unemployment leads the neo-ludditte movement to demand a halt to job automation. A "tax the machines" policy is implemented and a protracted struggle over what jobs get automated and who benefits/loses plays out across all of society (~60%)
2. AGI Race hits first. In this world, the US and China find themselves at war (or on the brink of it). Even if the US lets Taiwan get swallowed, the West is still going to gear up for the next fight. This means building as much as fast as possible. (~20%)
3. Alignment hits first. Some kind of alignment catastrophe happens and the world must deal with it. Maybe it is fatal, maybe it is just some self-replicating GPT-worm. In this world, the focus is on some sort of global AI governance to make sure that whatever the first Alignment Failure was (and given the way gov't works, totally ignoring other failure cases) (~10%)
4. Something wild. The singularity is supposed to be unpredictable. (~10%)