This just seems like an incredibly weak argument to me. A) it seems to me that prior research will be influenced much more than the probability for an arms race, because the first is more directly linked to public perception, B) we're mostly trying to spread awareness of the risk not the capability, and C) how do we even know that more awareness on the top political levels would lead to a higher probability for an arms race, rather than a higher probability for an international cooperation?
I feel like raising awareness has a very clear and fairly safe upside, while the downside is highly uncertain.
Why do you think this time is different to the nuclear arms race? The federation of atomic scientists didn't prevent it. It only slackened because russia ran ouf of steam.
Hi all,
So, as you may know, the first episode of Doctor Who, "Smile", was about a misaligned AI trying to maximize smiles (ish). And the latest, "Extremis", was about an alien race who instantiated conscious simulations to test battle strategies for invading the Earth, of which the Doctor was a subroutine.
I thought the common threat of AGI was notable, although I'm guessing it's just a coincidence. More seriously, though, this ties in with an argument I thought of, and want to know your take on: i
If we want to avoid an AI arms race, so that safety research has more time to catch up to AI progress, then we would want to prevent, if at all possible, these issues from becoming more mainstream. The reason is that if AGI in public perception becomes disassociated with Terminator (i.e. laughable, nerdy, and unrealistic) and more like a serious whoever-makes-this-first-can-take-over-the-world situation, then we will get an arms race faster.
I'm not sure I believe this argument myself. For one thing, being more mainstream has the benefit of attracting more safety research talent, government funding, etc. But maybe we shouldn't be spreading awareness without thinking this through some more.