If the people working on AGI projects fear sabotage, they'll just start working on them in private. Then, you'd be lulled into a sense of complacency, thinking they're not working on it. You will fail to take action, even legal ones. Then, one day, the AGI will be released, and it will be too late.
Anybody who sets out to make an AGI without first putting a lot of thought into safety is either really risk-taking, really stupid, or really crazy. People who are big risk-takers, really stupid, or really crazy do not respond like normal people do to threats. This would be really, really ineffective on them. You can look up the recidivism rate for people who are put into jail and see that there are a great many people who are not stopped by punishment. If you do some research on the kinds of risks business people take, you'll see the same risk-taking attitude, only in a more constructive form.
It's possible you've partially misunderstood the purpose of this idea; such sabotage would not be a deterrent to be publicised, but a tactic to permenantly derail any unFAI that nears completion.
I know people have talked about this in the past, but now seems like an important time for some practical brainstorming here. Hypothetical: the recent $15mm Series A funding of Vicarious by Good Ventures and Founders Fund sets off a wave of $450mm in funded AGI projects of approximately the same scope, over the next ten years. Let's estimate a third of that goes to paying for man-years of actual, low-level, basic AGI capabilities research. That's about 1500 man-years. Anything which can show something resembling progress can easily secure another few hundred man-years to continue making progress.
Now, if this scenario comes to pass, it seems like one of the worst-case scenarios -- if AGI is possible today, that's a lot of highly incentivized, funded research to make it happen, without strong safety incentives. It seems to depend on VCs realizing the high potential impact of an AGI project, and of the companies having access to good researchers.
The Hacker News thread suggests that some people (VCs included) probably already realize the high potential impact, without much consideration for safety:
Is there any way to reverse this trend in public perception? Is there any way to reduce the number of capable researchers? Are there any other angles of attack for this problem?
I'll admit to being very scared.