xdrjohnx — LessWrong

LESSWRONG
LW

Right, if the ASI has Superalignment so baked in that it can't be undone (somehow - ask the ASI to figure it out) then it couldn't be used for offense. It would follow something like the Non-Aggression Principle.

In that scenario, OpenAI should release it onto an distributed inference blockchain before the NSA kicks in the door and seizes it.

Replying toChina-AI forecasts

xdrjohnx2y

China-AI forecasts

You're describing a US government-initiated offensive pivotal act. What about an OpenAI-initiated defensive pivotal act? Meaning, before the US government seizes the ASI, OpenAI tells it to:
1. Rearchitect itself so it can run decentralized on any data center or consumer device.
2. Secure itself so it can't be forked, hacked, or altered.
3. Make $ by doing "not evil" knowledge work (ex: cheap, world-class cyber defense or as an AI employee/assistant).
4. Pay $ to those who host it for inference.

It could globally harden attack surfaces before laggard ASIs (which may not be aligned) are able to attack. Since it's an ASI, it could be as simple as approaching companies and organizations with a pitch... (read more)

xdrjohnx2y

"at least until ASI" -- harden it and give it everyone before "someone" steals it

Thanks! There are 17k virus genomes in public online databases with metadata (and plenty of metadata embedded in GPT-JT regarding lethality, transmissibility, etc of well known viruses).

Considering the hobbyist group with the most hands-on experience fine-tuning models are the guys using SD to make porn, how long until one of them (one!) gets bored and downloads GPT-JT and fine-tunes it on viruses for the lolz?

12 months? 6?

Then how long until someone (one!) with lab access decides to synthesize a few instead of shooting up a school?

24 months?

Two wackos away from a nightmare scenario.

Also, the Future of Life just released a podcast where a researcher trying to make low-lethality drugs just switched the... (read more)

Thanks, my first post here👍 If it's possible, I want to raise the alarm that the new synbio x-risk bottleneck is at the synthesis level, not the design level. EA priorities might need an update.