Q1a: Assuming beneficially political and economic development ...
2018/2025/2045
Q1b: Once our understanding of AI ...
Not difficult at all. It follows nearly automatically.
Q2a: Do you ever expect automated systems ...
Science is to important to be left to humans. Those systems will outperforms humans, of course. By a LARGE margin.
Q2b: To what extent does human engineering ...
Not to a great extent. Can be done from scratch if need be.
Q2c: What probability do you assign ...
10%/40%/99%
Q3a: Is it important to figure out how to make AI provably friendly
It will be no time to prove it mathematically in advance.
Q3b: What probability do ....
99% that humans will be wiped out. We may survive as non humans - 50%.
Q3c: How much money ...
Billion, maybe.
Q4: What is the current level of awareness ...
I am glad, that there is no mass hysteria about it.
Q5: ... Do possible risks from AI outweigh
Yes.
Q6: Can you think of any milestone...
Several.
Q7: How much have you read about the formal concepts ...
Quite a bit.
XiXiDu posted these questions for the purpose of getting feedback on how to revise them.
But since you answered the questions: Are you an AI expert? What is your full name? Is your CV available online?
I am emailing experts in order to raise and estimate the academic awareness and perception of risks from AI. Below are some questions I am going to ask. Please help to refine the questions or suggest new and better questions.
(Thanks goes to paulfchristiano, Steve Rayhawk and Mafred.)
Q1: Assuming beneficially political and economic development and that no global catastrophe halts progress, by what year would you assign a 10%/50%/90% chance of the development of artificial intelligence that is roughly as good as humans at science, mathematics, engineering and programming?
Q2: Once we build AI that is roughly as good as humans at science, mathematics, engineering and programming, how much more difficult will it be for humans and/or AIs to build an AI which is substantially better at those activities than humans?
Q3: Do you ever expect artificial intelligence to overwhelmingly outperform humans at typical academic research, in the way that they may soon overwhelmingly outperform humans at trivia contests, or do you expect that humans will always play an important role in scientific progress?
Q4: What probability do you assign to the possibility of an AI with initially (professional) human-level competence at general reasoning (including science, mathematics, engineering and programming) to self-modify its way up to vastly superhuman capabilities within a matter of hours/days/< 5 years?
Q5: How important is it to figure out how to make superhuman AI provably friendly to us and our values (non-dangerous), before attempting to build AI that is good enough at general reasoning (including science, mathematics, engineering and programming) to undergo radical self-modification?
Q6: What probability do you assign to the possibility of human extinction as a result of AI capable of self-modification (that is not provably non-dangerous, if that is even possible)?