[Speaking in character] I'm held responsible for building this nation's trade, for protecting it from those who would be our enemies from abroad, all while deftly managing potential threats from my own political rivals at home. I am very busy. My staff knows better than to bother me with fevered dreams of a few science fiction devotees.
It's different when some crazy group or ideology starts exercising political power. If some foreign cult starts lobbying its government or to reduce my people to serfs, or trying to take power itself, I will pay attention. That is a threat I can recognize, having seen it repeated in history again and again.
Obviously, I don't want open war with a great power if it can be avoided. So I would instruct my diplomats and the public relations staff (who used to be called the propaganda ministry a long time ago) to initiate a subtle campaign to portray this strange science fiction group as a dangerous cult, mad for power. Of course they want to have a monopoly over computing power. I would embarrass their host government, suggesting that we smaller countries cannot help to but see the growing prominence of this group as a mounting threat of imperialist conquest. I would want assurances that, even if these people are allowed to remain free and spout their venom, none of them will be regarded as respectable or ever hold a position of authority.
If I were persuaded there really is "no way around large-scale politico-military struggle." I would consider this thing they call "cyber-warfare" as a form of asymmetric resistance, in addition to training my conventional forces to repel a larger invader. I hope we would win. But if our defeat looks imminent, I would make preparations to see that my nation's technical infrastructure could survive underground in the event of a foreign occupation. We do not relish the idea of being reduced to nothing more than agricultural peasants. If it's our computers they want, then we will guard our computers all the more.
I agree that there is likely no risk if nobody takes the SIAI seriously. But if at some point a powerful entity does take it serious, especially the country where most of the SIAI members reside in, then the best case scenario might be that it will be put under government control. I just don't see that a government that does take fooming AI serious would trust the SIAI in trying to implement friendliness, or would allow them to do it even if they believed so.
Having all known life on Earth concentrated on a single planet is an existential risk. So we should try to spread out, right? As soon as possible?
Yet, if we had advanced civilizations on two planets, that would be two places for unfriendly AI to originate. If, as many people here believe, a single failed trial ruins the universe, you want to have as few places trying it as possible. So you don't want any space colonization until after AI is developed.
If we apply that logic to countries, you would want as few industrialized nations as possible until AAI (After AI). So instead of trying to help Africa, India, China, and the Middle East develop, you should be trying to suppress them. In fact, if you really believed the calculations I commonly see used in these circles about the probability of unfriendly AI and its consequences, you should be trying to exterminate human life outside of your developed country of choice. Failing to would be immoral.
And if you apply it within the USA, you need to pick one of MIT and Stanford and Carnegie Mellon, and burn the other two to the ground.
Of course, doing this will slow the development of AI. But that's a good thing, if UFAI is most likely and has zero utility.
In fact, if slowing development is good, probably the best thing of all is just to destroy civilization and stop development completely.
Do you agree with any of this? Is there a point where you think it goes too far? If so, say where it goes too far and explain why.
I see two main flaws in the reasoning.
ADDED: A number of the comments so far imply that the first AI built will necessarily FOOM immediately. FOOM is an appealing argument. I've argued in favor of it myself. But it is not a theorem. I don't care who you are; you do not know enough about AI and its future development to bet the future of the universe on your intuition that non-FOOMing AI is impossible. You may even think FOOM is the default case; that does not make it the only case to consider. In this case, even a 1% chance of a non-foom AI, multiplied by astronomical differences in utility, could justify terrible present disutility.