Draft of Muehlhauser & Salamon, 'Intelligence Explosion: Evidence and Import'

lukeprog

Anna Salamon and I have finished a draft of "Intelligence Explosion: Evidence and Import", under peer review for The Singularity Hypothesis: A Scientific and Philosophical Assessment (forthcoming from Springer).

Your comments are most welcome.

Edit: As of 3/31/2012, the link above now points to a preprint.

Your comments are most welcome.

Edit: As of 3/31/2012, the link above now points to a preprint.

It's not an intelligence explosion if humans are kept in the loop.

(You could argue that e.g. the Flash Crash resulted because humans weren't in the loop, but humans can still interrupt the process eventually, so they're not really out of the loop - they just respond more slowly.)

Is this so?

It seems to me there's a continuum between "humans carefully monitoring and controlling a weakish AI system" and "superintelligent AI-in-a-box cleverly manipulates humans in order to wreak havoc". It seems that as the world transitions from one to the other, at some point it will pass an "intelligence explosion" threshold. But I don't think that it ever passes a "humans are no longer in the loop" threshold.

7Dreaded_Anomaly14y

A new paper suggests that ultrafast machine trading is causing crashes that don't last long enough for humans to react: [...] If we want oversight of this kind of trading, it seems like we'll have to rely on more ultrafast machines.

4timtyler14y

There is no "loop", there are many loops, some of which humans have already been eliminated from - through the conventional process of automation. Humans being in some of the loops does not necessarily even slow things down very much - if the other loops are permitted to whir around at full speed. In my essay on the topic I cite increasingly infrequent periodic code reviews as an example of how human influence on designing the next generation of dominant creatures could fade away gradually, without causing very much slow-down. That sort of thing might result in a "slightly muffled" explosion, but it would still be an explosion.

26

Draft of Muehlhauser & Salamon, 'Intelligence Explosion: Evidence and Import'

26

26

26

Draft of Muehlhauser & Salamon, 'Intelligence Explosion: Evidence and Import'

26

26