Eliezer Yudkowsky says that, "Exponentials are Kurzweil's thing. They aren't dangerous."
Different people who believe in some form of Singularity disagree on the specifics. By trying to capture every view, I fear I have mangled them all.
Your hypothesis seems to include itself as a premise? Is this correct? I am sorry that I have to ask this, I lack a lot of education :-(
If you define "Singularity" as "an AI going to superintelligence quickly" then yeah, it does, and that shouldn't be a premise. I was defining "Singularity" as "a massive change to the world as we know it, probably resulting in something either very awesome or very horrible.
I am sorry, this sounds a bit like, "the world will end a few years or less after the first antimatter asteroid has been detected to be on a collision course with earth". Maybe it is just my complete lack of training in matters of rationality that makes me think so. I am really sorry in that case :-(
To people who believe that there will be a Singularity, it does sound like that. Some people believe that smarter-than-human AI is impossible or that it will not cause massive change to the world as we know it. Again, I appear to be using a different definition from you: if one defines a Singularity as a smarter-than-human AI, I was being tautological.
Does this mean that a hypothesis, or prediction, does not need to be specific about its possible timeframe? We just have to wait? At what point do we then decide to turn to other problems? Maybe I am completely confused here, but how do you update your risk estimations if you can't tell when a risk ceases to be imminent?
I don't know enough AI science to answer this question completely. I don't know what would be strong evidence that human level AI or higher is impossible, other than the brain turning out to be non-Turing-computable. If a human level or slightly smarter AI is developed and it does not self-improve further (or enough to drastically change the world), this would be evidence against a hard takeoff or a Singularity.
Since, as far as I can tell, in your hypothesis, smarter-than-human AI is strongly correlated with the occurrence of a Singularity, would it be reasonable to name some concrete conditions required to enable such a technology?
Other than "a good enough understanding of mind to formulate an AI and enough computing power to run it" I really don't know enough to say. Will someone with more knowledge of AI please fill in the gaps in my explanation?
To be clear, I am just trying to figure out how the proponents of explosive recursive self-improvement can be surprised by data.
A proponent of explosive recursive self-improvement can be surprised by an AI of human intelligence or slightly greater that does not go FOOM. Or by finding out that AI is in principle impossible (though proving something in principle impossible is very hard).
Maybe this is perfectly clear for everyone else, I am sorry, I don't know where else to ask about this.
This is the right place, but I'm not the best person. Again, I'd love for somebody who knows some AI to help with the questions I couldn't answer.
Today's post, Beware the Unsurprised was originally published on May 3, 2007. A summary (from the LW wiki):
Discuss the post here (rather than in the comments of the original post).
This post is part of a series rerunning Eliezer Yudkowsky's old posts so those interested can (re-)read and discuss them. The previous post was Think like Reality, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it, posting the next day's sequence reruns post, summarizing forthcoming articles on the wiki, or creating exercises. Go here for more details, or to discuss the Sequence Reruns.