Originally posted to SecondBest.ca ; Zvi responds here. I. Oversight of AGI labs is prudent 1. It is in the U.S. national interest to closely monitor frontier model capabilities. 2. You can be ambivalent about the usefulness of most forms of AI regulation and still favor oversight of the frontier...
The risks from developing superintelligent AI are potentially existential, though hard to visualize. A better approach to communicating the risk is to illustrate the dangers from existing systems, and to then imagine how those dangers will increase as AIs become steadily more capable. An AGI could spontaneously develop harmful goals...
Originally posted to SecondBest.ca ; Zvi responds here.
Can read about quantization and the compression of GPT models here:
https://beuke.org/quantization/
The risks from developing superintelligent AI are potentially existential, though hard to visualize. A better approach to communicating the risk is to illustrate the dangers from existing systems, and to then imagine how those dangers will increase as AIs become steadily more capable.
An AGI could spontaneously develop harmful goals and subgoals, or a human could simply ask an obedient AGI to do things that are harmful. This second bucket requires many fewer assumptions about arcane results in decision theory. And even if those arcane results are where the real x-risk lies, it’s easier to build an intuition for the risks by working from the bottom-up, as scenarios in which “AGI gets an... (read 378 more words →)
