I know people have talked about this in the past, but now seems like an important time for some practical brainstorming here. Hypothetical: the recent $15mm Series A funding of Vicarious by Good Ventures and Founders Fund sets off a wave of $450mm in funded AGI projects of approximately the same scope, over the next ten years. Let's estimate a third of that goes to paying for man-years of actual, low-level, basic AGI capabilities research. That's about 1500 man-years. Anything which can show something resembling progress can easily secure another few hundred man-years to continue making progress.
Now, if this scenario comes to pass, it seems like one of the worst-case scenarios -- if AGI is possible today, that's a lot of highly incentivized, funded research to make it happen, without strong safety incentives. It seems to depend on VCs realizing the high potential impact of an AGI project, and of the companies having access to good researchers.
The Hacker News thread suggests that some people (VCs included) probably already realize the high potential impact, without much consideration for safety:
...I think this exactly the sort of innovation timeline real venture capitalists should be considering - funding real R&D that could have a revolutionary impact even if the odds are against it.
The company to get all of this right will be the first two trillion dollar company.
Is there any way to reverse this trend in public perception? Is there any way to reduce the number of capable researchers? Are there any other angles of attack for this problem?
I'll admit to being very scared.
Sabotage would not work. Several reasons:
If the people working on AGI projects fear sabotage, they'll just start working on them in private. Then, you'd be lulled into a sense of complacency, thinking they're not working on it. You will fail to take action, even legal ones. Then, one day, the AGI will be released, and it will be too late.
Anybody who sets out to make an AGI without first putting a lot of thought into safety is either really risk-taking, really stupid, or really crazy. People who are big risk-takers, really stupid, or really crazy do not respond like normal people do to threats. This would be really, really ineffective on them. You can look up the recidivism rate for people who are put into jail and see that there are a great many people who are not stopped by punishment. If you do some research on the kinds of risks business people take, you'll see the same risk-taking attitude, only in a more constructive form.
People who previously didn't know anything about the AGI project would view the AGI company as the good guys and the saboteurs as the bad guys. Whenever violence gets involved, opinions polarize, permanently. The saboteurs would find it 10,000 times harder just to exist, they'd lose a huge amount of support, making it 10,000 times harder to exist again. MUCH WORSE would be the biasing effects of "stimulus generalization". That is the psychological effect that causes people to feel prejudiced against, say Middle Easterners in general because they fear terrorists. If those who want to protect the world from risky AGI begin to sabotage projects, public opinion may just round off and lump everybody warning against risky AGI under the "terrorist" label. They might feel such a strong bias that they assume anyone warning about unfriendly AGI is a psycho and ignore the danger of unfriendly AGI completely
There will be numerous projects in numerous places at numerous times. It wouldn't be a matter of stopping just one project, or projects in just one country. The challenge here is to make ALL the projects in the entire WORLD safe, and not just the ones we have now but ALL the ones that might be there for forever into the future. Whatever it is that we do to increase the safety of this research, it would have to affect ALL of the projects. There's no way in hell anybody could be THAT effective at sabotage. All three of the ideas I came up with to prevent risky AGI have the potential to scale to the rest of the world and last through that indefinite time period.
Most people experience optimism bias, it's a common bias. People think "Oh look something dangerous happened over there. It won't happen to me." I have observed that upper class people often have especially strong optimism bias. I think this is because many of them have led lives that were very sheltered and privileged, and since that type of life frequently results in a dearth of the sort of "wake up calls" that cause one to challenge more instances of optimism bias, they frequently act as if nothing can go wrong. I can see them easily ignoring the risk of sabotage, or paying more for security and assuming that means they are safe.
Sabotage would not prevent risky AGIs from being created. Period. And it could make things harder. If you really want to do something about dangerous AGI, put all that energy into spreading the word about the dangers.
See Also Three Possible Solutions