XiXiDu comments on Existential Risk and Public Relations - Less Wrong

36 Post author: multifoliaterose 15 August 2010 07:16AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (613)

Comments (613)

You are viewing a single comment's thread. Show more comments above.

Comment author: XiXiDu 19 November 2010 10:15:20AM *  1 point [-]

For what it's worth, Eliezer has been influential/persuasive enough to get the SIAI created and funded despite having absolutely no academic qualifications. He's also responsible for coining "Seed AI".

Interesting, when did he come up with the concept of "Seed AI". Because it is mentioned in Karl Schroeder's Ventus (Tor Books, 2000.) ISBN 978-0312871970.

Comment author: Risto_Saarelma 19 November 2010 12:11:31PM *  1 point [-]

Didn't find the phrase "Seed AI" there. One plot element is a "resurrection seed", which is created by an existing, mature evil AI to grow itself back together in case it's main manifestation is destroyed. A Seed AI is a different concept, it's something the pre-AI engineers put together that grows into a superhuman AI by rewriting itself more and more powerful. A Seed AI is specifically a method to get to AGI from not having one, not just an AI that grows from a seed-like thing. I don't remember recursive self-improvement being mentioned with the seed in Ventus.

A precursor concept where the initial AI bootstraps itself by merely learning things, not necessarily by rewriting it's own architecture, goes all the way back to Alan Turing's 1950 paper on machine intelligence.

Comment author: XiXiDu 19 November 2010 12:36:58PM *  1 point [-]

Here is a quote from Ventus:

Look at it this way. Once long ago two kinds of work converged. We'd figured out how to make machines that could make more machines. And we'd figured out how to get machines to... not exactly think, but do something very much like it. So one day some people built a machine which knew how to build a machine smarter than itself. That built another, and that another, and soon they were building stuff the men who made the first machine didn't even recognize.

[...]

And, some of the mechal things kept developing, with tremendous speed, and became more subtle than life. Smarter than humans. Conscious of more. And, sometimes, more ambitious. We had little choice but to label them gods after we saw what they could do--namely, anything.

Comment author: timtyler 20 November 2010 12:58:29PM *  1 point [-]

...and here's a quote from I.J. Good, from 1965:

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make.

He didn't coin the term "Seed AI" either.

Comment author: XiXiDu 20 November 2010 01:01:37PM *  0 points [-]

Yes, but I believe it is a bit weird for a Wikipedia article to state that someone is the originator of the Seed AI theory when he just coined the term. I wasn't disputing anything, just trying to figure out if it is actually the case that Yudkowsky came up with the concept in the first place.

Comment author: timtyler 20 November 2010 01:15:56PM *  -1 points [-]

Not the concept - the term.

"Seed AI theory" probably refers to something or another in here - which did indeed originate with Yu'El.

Presumably http://en.wikipedia.org/wiki/Seed_AI should be considered to be largely SIAI marketing material.

Comment author: XiXiDu 19 November 2010 12:34:30PM 0 points [-]

They did not command the wealth of nations, these researchers. Although their grants amounted to millions of Euros, they could never have funded a deep-space mission on their own, nor could they have built the giant machineries they conceived of. In order to achieve their dream, they built their prototypes only in computer simulation, and paid to have a commercial power satellite boost the Wind seeds to a fraction of light speed. [...] no one expected the Winds to bloom and grow the way they ultimately did.

It is further explained that the Winds were designed to evolve on their own so they are not mere puppets of human intentions but possess their own intrinsic architecture.

In other places in the book it is explained how humans did not create their AI Gods but that they evolved themselves from seeds designed by humans.

Powered by Reddit Powered by Reddit