Major update here.

Related to: Should I believe what the SIAI claims?

Reply to: Ben Goertzel: The Singularity Institute's Scary Idea (and Why I Don't Buy It)

... pointing out that something scary is possible, is a very different thing from having an argument that it’s likely. — Ben Goertzel

What I ask for:

I want the SIAI or someone who is convinced of the Scary Idea¹ to state concisely and mathematically (and with possible extensive references if necessary) the decision procedure that led they to make the development of friendly artificial intelligence their top priority. I want them to state the numbers of their subjective probability distributions² and exemplify their chain of reasoning, how they came up with those numbers and not others by way of sober calculations.

The paper should also account for the following uncertainties:

Comparison with other existential risks and how catastrophic risks from artificial intelligence outweigh them.
Potential negative consequences³ of slowing down research on artificial intelligence (a risks and benefits analysis).
The likelihood of a gradual and controllable development versus the likelihood of an intelligence explosion.
The likelihood of unfriendly AI⁴ versus friendly and respectively abulic⁵ AI.
The ability of superhuman intelligence and cognitive flexibility as characteristics alone to constitute a serious risk given the absence of enabling technologies like advanced nanotechnology.
The feasibility of “provably non-dangerous AGI”.
The disagreement of the overwhelming majority of scientists working on artificial intelligence.
That some people who are aware of the SIAI’s perspective do not accept it (e.g. Robin Hanson, Ben Goertzel, Nick Bostrom, Ray Kurzweil and Greg Egan).
Possible conclusions that can be drawn from the Fermi paradox⁶ regarding risks associated with superhuman AI versus other potential risks ahead.

Further I would like the paper to include and lay out a formal and systematic summary of what the SIAI expects researchers who work on artificial general intelligence to do and why they should do so. I would like to see a clear logical argument for why people working on artificial general intelligence should listen to what the SIAI has to say.

Examples:

Here are are two examples of what I'm looking for:

Am I A Sim?
Asteroid Deflection as a Public Good

The first example...