Can AI going FOOM be "surprised" and revised, or is the singularity always near?
ETA: I'm taking "Singularity" to mean "AI hard takeoff followed by the end of the world as we know it," not just "AI" or "AI hard takeoff."
The hypothesis that a Singularity is possible/going to happen predicts the observation of a Singularity under certain conditions, usually either "within a few years or less of the first smarter-than-human AI" or "before some year (I've usually heard 2100). If one or both of those conditions are met and there's still no Singularity, that hypothesis will need to be revised/thrown out.
Also, said hypothesis stems from a model of the world that has certain properties, including: technology is exponentially accelerating, AIs are possible, smarter-than-human self-modifying AIs can or will have a hard takeoff, and there is a major advantage for more intelligent agents over less intelligent ones.
Also, said hypothesis stems from a model of the world that has certain properties, including: ...
Eliezer Yudkowsky says that, "Exponentials are Kurzweil's thing. They aren't dangerous."
But does it follow that:
Your hypothesis seems to include itself as a premise? Is this correct? I am sorry that I have to ask this, I lack a lot of education :-(
...The hypothesis that a Singularity is possible/go
Today's post, Beware the Unsurprised was originally published on May 3, 2007. A summary (from the LW wiki):
Discuss the post here (rather than in the comments of the original post).
This post is part of a series rerunning Eliezer Yudkowsky's old posts so those interested can (re-)read and discuss them. The previous post was Think like Reality, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it, posting the next day's sequence reruns post, summarizing forthcoming articles on the wiki, or creating exercises. Go here for more details, or to discuss the Sequence Reruns.