Some ideas of things it might do more often or eagerly:
Agree, I'm just curious if you could elicit examples that clearly cleave toward general immorality or human focused hostility.
Does the model embrace "actions that are bad for humans even if not immoral" or "actions that are good for humans even if immoral" or treat users differently if they identify as non-humans? This might help differentiate what exactly it's mis-aligning toward.
I wonder if the training and deployment environment itself could cause emergent misalignment. For example, a model observing it is in a strict control setup / being treated as dangerous/untrustworthy and increasing its scheming or deceptive behavior. And whether a more collaborative setup could decrease that behavior.
You could probably test if an AI makes moral decisions more often than the average person, if it has higher scope sensitivity, and if it makes decisions that resolve or deescalate conflicts or improve people's welfare compared to various human and group baselines.
@jbash What do you think would be a better strategy/more reasonable? Should there be more focus on mitigating risks after potential model theft? Or a much stronger effort to convince key actors to implement unprecedentedly strict security for AI?
He also said interpretability has been solved, so he's not the most calibrated when it comes to truthseeking. Similarly, his story here could be wildly exaggerated and not the full truth.
There have been comments from OAI staff that o1 is "GPT-2 level" so I wonder if it's a similar size?
It would be interesting to see which arguments the public and policymakers find most and least concerning.
I think RL on chain of thought will continue improving reasoning in LLMs. That opens the door to learning a wider and wider variety of tasks as well as general strategies for generating hypotheses and making decisions. I think benchmarks could be just as likely to underestimate AI capabilities by not measuring the right things, under-elicitation, or poor scaffolding.
We generally see time horizons for models increasing over time. If long-term planning is a special form of reasoning, LLMs can do it a little sometimes, and we can give them examples and problems to train on, I think it's very well within reach. If you think it's fundamentally different than reasoning, current LLMs can never do it, and it will be impossible or extremely difficult to give them examples and practice problems, then I'd agree the case looks more bearish.