Survey: Risks from AI

XiXiDu

Related to: lesswrong.com/lw/fk/survey_results/

I am currently emailing experts in order to raise and estimate the academic awareness and perception of risks from AI and ask them for permission to publish and discuss their responses. User:Thomas suggested to also ask you, everyone who is reading lesswrong.com, and I thought this was a great idea. If I ask experts to publicly answer questions, to publish and discuss them here on LW, I think it is only fair to do the same.

Answering the questions below will help the SIAI and everyone interested to mitigate risks from AI to estimate the effectiveness with which the risks are communicated.

Questions:

Assuming no global catastrophe halts progress, by what year would you assign a 10%/50%/90% chance of the development of human-level machine intelligence? Feel free to answer 'never' if you believe such a milestone will never be reached.
What probability do you assign to the possibility of a negative/extremely negative Singularity as a result of badly done AI?
What probability do you assign to the possibility of a human level AGI to self-modify its way up to massive superhuman intelligence within a matter of hours/days/< 5 years?
Does friendly AI research, as being conducted by the SIAI, currently require less/no more/little more/much more/vastly more support?
Do risks from AI outweigh other existential risks, e.g. advanced nanotechnology? Please answer with yes/no/don't know.
Can you think of any milestone such that if it were ever reached you would expect human‐level machine intelligence to be developed within five years thereafter?

Note: Please do not downvote comments that are solely answering the above questions.

Related to: lesswrong.com/lw/fk/survey_results/

Answering the questions below will help the SIAI and everyone interested to mitigate risks from AI to estimate the effectiveness with which the risks are communicated.

Questions:

Assuming no global catastrophe halts progress, by what year would you assign a 10%/50%/90% chance of the development of human-level machine intelligence? Feel free to answer 'never' if you believe such a milestone will never be reached.
What probability do you assign to the possibility of a negative/extremely negative Singularity as a result of badly done AI?
What probability do you assign to the possibility of a human level AGI to self-modify its way up to massive superhuman intelligence within a matter of hours/days/< 5 years?
Does friendly AI research, as being conducted by the SIAI, currently require less/no more/little more/much more/vastly more support?
Do risks from AI outweigh other existential risks, e.g. advanced nanotechnology? Please answer with yes/no/don't know.
Can you think of any milestone such that if it were ever reached you would expect human‐level machine intelligence to be developed within five years thereafter?

Note: Please do not downvote comments that are solely answering the above questions.

define "global catastrophe halts progress"
probability of what exactly conditional on what exactly?
probability of what exactly conditional on what exactly?
define "require"
define "outweigh"

ETA: Since multiple people seem to find this comment objectionable for some reason I don't understand, let me clarify a little. For 1 it would make some difference to my estimate whether we're conditioning on literal halting of progress or just significant slowing, and things like how global the event needs to be. (This is a relatively minor ambiguity, but 90th percentiles can be pretty sensitive to such things.) For 2 it's not clear to me whether it's asking for the probability that a negative singularity happens conditional on nothing, or conditional on no disaster, or conditional on badly-done AI, or whether it's asking for the probability that it's possible that such a singularity will happen. All these would have strongly different answers. For 3 something similar. For 4 it's not clear whether to interpret "require" as "it would be nice", or "it would be the best use of marginal resources", or "without it there's essentially no chance of success", or something else. For 5 "outweigh" could mean outweigh in probability or outweigh in marginal value of risk reduction, or outweigh in expected negative value, or something else.

P(human-level AI by ? (year) | no wars ∧ no natural disasters ∧ beneficially political and economic development) = 10%/50%/90%/0%
P(negative Singularity | badly done AI) = ?; P(extremely negative Singularity | badly done AI) = ? (where 'negative' = human extinction; 'extremely negative' = humans suffer;).
P(superhuman intelligence within hours | human-level AI on supercomputer with Internet connection) = ?; P(superhuman intelligence within days | human-level AI on supercomputer with Internet connection) = ?; P(superhuman intelligence within < 5 years |

... (read more)

13

Survey: Risks from AI

13

13

13

Survey: Risks from AI

13

13