Coincidentally, that scene in The Big Short takes place on January 11 (2007) :D
I read it as a joke, lol.
https://www.lesswrong.com/posts/jnyTqPRHwcieAXgrA/finding-goals-in-the-world-model
Could it be possible to poison the world model an AGI is based on to cripple its power?
Use generated text/data to train world models based on faulty science like miasma, phlogiston, ether, etc.
Remove all references to the internet or connectivity based technology.
Create a new programming language that has zero real world adoption, and use that for all code based data in the training set.
There might be a way to elicit how aligned/unaligned the putative AGI is.
Does there have to be a reward? This is using brute force to create the underlying world model. It's just adjusting weights right?
Brute force alignment by adding billions of tokens of object level examples of love, kindness, etc to the dataset. Have the majority of humanity contribute essays, comments, and (later) video.
I wonder what kind of signatures a civilization gives off when AGI is nascent.
Develop a training set for alignment via brute force. We can't defer alignment to the ubernerds. If enough ordinary people (millions? tens of millions?) contribute billions or trillions of tokens, maybe we can increase the chance of alignment. It's almost like we need to offer prayers of kindness and love to the future AGI: writing alignment essays of kindness that are posted to reddit, or videos extolling the virtue of love that are uploaded to youtube.
AI presents both staggering opportunity and chilling peril. Developing intelligent machines could help eradicate disease, poverty, and hunger within our lifetime. But uncontrolled AI could spell the end of the human race. As Stephen Hawking warned, "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks."
AI Safety is a group project for us all. We need everyone to participate - the ESFPs to the INTJs!
Capturing the essence and subtleties of core values needs input across a broad span of humanity.
Assumption 1 - large language models will be the basis of AGI.
Assumption 2 - One way to add the abstraction of a value like "kindness is good" into the model is to add a large corpus of written material on Kindness during training (or retraining).
The Kindness Project is a website with a prompt, like a college essay. Users add their stories to the open collection based on the prompt: "Tell a story about how you impacted or were... (read more)
There is a problem in that any group that is generating alpha would likely lose alpha/person if they allow random additional people into their group.
Think Renaissance Medallion fund. It's been closed to outside investment since near its inception 30 years ago. Prerequisites for the average person joining would be something like true-genius level Phd in a related STEM field.
An analogue which is closely related is poker players who use solvers to improve their game. The starting stakes are a bit lower. The solvers are like a few thousand dollars + equipment to run them, a class on how to use them runs a similar couple thousand bucks, and then there is the... (read more)