JStewart comments on A Primer On Risks From AI - Less Wrong

15 Post author: XiXiDu 24 March 2012 02:32PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (33)
Tags: ai risk intelligence explosion

Comments (33)

You are viewing a single comment's thread.

Comment author: JStewart 24 March 2012 05:31:10PM 3 points [-]

I like the combination of conciseness and thoroughness you've achieved with this.

There are a couple of specific parts I'll quibble about:

Therefore the next logical step is to use science to figure out how to replace humans by a better version of themselves, artificial general intelligence.

"The Automation of Science" section seems weaker to me than the others, perhaps even superfluous. I think the line I've quoted is the crux of the problem; I highly doubt that the development of AGI will be driven by any such motivations.

Will we be able to build an artificial general intelligence? Yes, sooner or later.

I assign a high probability to the proposition that we will be able to build AGI, but I think a straight "yes" is too strong here.

Comment author: jmmcd 27 March 2012 09:54:38AM *  0 points [-]

Agreed -- AGI will probably not be developed with the aim of improving science.

I also want to quibble about this:

Therefore the next logical step is to use science to figure out how to replace humans by a better version of themselves

Since most readers don't want to be replaced, at least in one interpretation of that term, this line sticks in the throat and breaks the flow. The natural response is something like "logical? According to whose goals?"

Powered by Reddit Powered by Reddit