XiXiDu comments on Holden Karnofsky's Singularity Institute Objection 3 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (8)
Assume you were to gradually transform Google Maps into a seed AI, at what point would it become an existential risk and how? And why wouldn't you just skip that step?
More here.
If it tries to self-improve, and as a side effect turns the universe to computronium.
If it gains a general intelligence, and as a part of trying to provide better search results, it realizes that self-modification could bring much faster search results.
This whole idea of a harmless general intelligence is just imagining a general intelligence which is not general enough to be dangerous; which will be able to think generally, and yet somehow this ability will always reliably stop before thinking something that might end bad.
Thanks, I completely missed that. Explains a lot.
That reminds me of Project Pigeon, only with a weapon capable of destroying the planet, and we're the pigeon.
A very important part of Google Maps is Street View, which is created by cars driving around and taking pictures of everything. These could be viewed as 'arms' of the seed AI, along with its surveillance satellites, WiFi sniffing for more accurate geolocation, 3D modelling of buildings, and the recently introduced building-interior maps.
Which is to say, Super Google Maps could be a gigantic surveillance network and pervasive examiner of every corner of reality so it could be as up to date as possible.
How does one do a gradual transformation on a discontinuous space such as the space of computer programs that are somehow related to navigation or general intelligence?