Firstly, if I knew for a fact that at least one Singularity-grade AI existed, then I would believe that the creation of another such AI is virtually inevitable. The question is not whether another such AI would be created, but when.
But the "when" question can be rather open-ended; in order to answer it, I would collect evidence to assess the relative capabilities of all the major organizations who could create such an AI, and compare them to the capabilities of my own organization.
By analogy, the USA was the first nation on Earth to develop nuclear weapons. However, even though nuclear proliferation became inevitable as soon as the first nuke was developed, other nations got their nukes at different times; some nations are still working on theirs. Superpowers got their nukes first, followed by other, less powerful nations. I can easily imagine the development of transhuman AI following the same pattern (assuming, of course, that the AIs stay in their boxes, somehow).
Edited to clarify: As per my previous comment, I consider that once a boxed transhuman AI is created, it will inevitably break out, assuming that it can interact with any system that can cause it to be let out of the box -- which includes human gatekeepers.
So if you're confronted with an AI that might or might not be friendly, you will think it inevitable that an AI will be released, but think the AI you're talking to wouldn't be able to use that fact to somehow persuade you to let it out?
Assume you're fairly certain but not positive that the AI you're talking to is unfriendly, and that you are in moderately optimal organization for this task - say something like MIT or the government of France, something the size of other organizations that could do the same thing, and less able and motivated to complete this project than a few dozen other entities.
Some of you have expressed the opinion that the AI-Box Experiment doesn't seem so impossible after all. That's the spirit! Some of you even think you know how I did it.
There are folks aplenty who want to try being the Gatekeeper. You can even find people who sincerely believe that not even a transhuman AI could persuade them to let it out of the box, previous experiments notwithstanding. But finding anyone to play the AI - let alone anyone who thinks they can play the AI and win - is much harder.
Me, I'm out of the AI game, unless Larry Page wants to try it for a million dollars or something.
But if there's anyone out there who thinks they've got what it takes to be the AI, leave a comment. Likewise anyone who wants to play the Gatekeeper.
Matchmaking and arrangements are your responsibility.
Make sure you specify in advance the bet amount, and whether the bet will be asymmetrical. If you definitely intend to publish the transcript, make sure both parties know this. Please note any other departures from the suggested rules for our benefit.
I would ask that prospective Gatekeepers indicate whether they (1) believe that no human-level mind could persuade them to release it from the Box and (2) believe that not even a transhuman AI could persuade them to release it.
As a courtesy, please announce all Experiments before they are conducted, including the bet, so that we have some notion of the statistics even if some meetings fail to take place. Bear in mind that to properly puncture my mystique (you know you want to puncture it), it will help if the AI and Gatekeeper are both verifiably Real People<tm>.
"Good luck," he said impartially.