All of Bleys's Comments + Replies

I agree that benchmarks might not be the right criteria, but training cost isn't the right metric either IMO, since compute and algorithmic improvement will be bringing these costs down every year. Instead, I would propose an effective compute threshold, i.e. number of FLOP while accounting for algorithmic improvements.

On further reflection, I'd tentatively propose something along these lines as an additional measure: As I've now seen others suggest, trigger limits determined only as a percentage of the state of the art's performance. This could be implemented as a proposal to give a government agency the power to work as the overseer and final arbiter of deciding, once per year for the following year (and ad-hoc on an emergency basis), the metrics and threshold percentages of indexing what is determined state of the art. This would be done in consultation with representatives from each of the big AI labs (as determined by, e.g., having invested >$100M in AI compute), and including broader public, academic, and open source AI community feedback but ultimately decided by the agency. The power could also be reserved for the agency to determine that specific model capabilities, if well defined and clearly measurable, could be listed as automatically triggering regulation. This very clearly makes the regulation target the true "frontier AI" while leaving others out of the collateral crosshairs. I say tentatively, as an immediate need for any sort of specific model-capability-level regulation to prevent existential risk is not remotely apparent with the current architectures for models (Autoregressive LLMs). I see the potential in the future for risk, but pending major breakthroughs in architecture. Existing models, and the immediately coming generation, are trivially knowable as non-threatening at an existential level. Why? They are incapable of objective driven actions and planning. The worst that can be done is within the narrow span of agent-like actions that can be covered via extensive and deliberate programmatic connection of LLMs into heavily engineered systems. Any harms that might result would be at worst within a narrow scope that's either tangential to the intended actions, or deliberate human intent that's likely covered within existing criminal frameworks. The worst

This sounds an awful lot like one of the examples I gave above. Ok, so you're focused on "risk reduction" and "reducing akrasia." So what's that mean? You've decided to buckle-up, wear sunscreen, and not be so lazy? Can't I get that from Reader's Digest or my mom?