private_messaging comments on Holden's Objection 1: Friendliness is dangerous - Less Wrong

11 Post author: PhilGoetz 18 May 2012 12:48AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (428)

You are viewing a single comment's thread.

Comment author: private_messaging 27 May 2012 03:22:09PM *  0 points [-]

The much stronger issue he raised is that it may well be that outside imagination and fiction, there is no monolithic 'intelligence' thing, and the 'benevolent ruler of the earth' software is then more dangerous than e.g. software that uses search and hill climbing to design better microchips, or design cures for diseases, or the like, without being 'intelligent' in the science fictional sense, and while lacking any form of real world volition. The "benevolent ruler of the earth" software would then, also, fail to provide any superior technical solutions to our problems, as this "intelligence" does not bring any important advantage over the algorithms normally used for problem solving.

The chip improver would spit out the blueprints, the cure designer would spit out the projected molecular images and DNA sequences, etc - no oracle crap with the 'utility' of making people understand something, which appears both near-impossible and entirely unnecessary.

Comment author: CuSithBell 27 May 2012 03:41:52PM 1 point [-]

Outside of mystic circles, it is fairly uncontroversial that it is in principle possible to construct out of matter an object capable of general intelligence. Proof is left to the reader.