Rain comments on A Primer On Risks From AI - Less Wrong

15 Post author: XiXiDu 24 March 2012 02:32PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (33)

You are viewing a single comment's thread. Show more comments above.

Comment author: Rain 25 March 2012 12:05:50PM *  0 points [-]

Unless it develops better quantum computing or exploits other strange physical phenomena we don't know about.

Comment author: Dmytry 25 March 2012 12:37:39PM *  -1 points [-]

At which time all bets are off with regards to whenever it would even compete for resources. A lot of stuff of this kind can happen, e.g. discovering that we are in simulation.

The bottom line is, now that we established limitations, the AI risks better be prefaced with "If AI develops awesome quantum computing, but it still needs atoms from your body, the following theoretizations about godlike AIs might apply:".

Furthermore, as the AI has to start off with the low hanging fruit - optimizing itself on commodity hardware - the argumentation is not about random points in the awesome quantum mind design space, but our original classical AI's creations, subject to original AI getting perfectly ordinary cold feet about the transition due to wide variety of heuristics that scream 'no, that change is too big and unpredictable', and the AI trying to preserve itself through the transition.