Stuart_Armstrong comments on Reduced impact AI: no back channels - Less Wrong

13 Post author: Stuart_Armstrong 11 November 2013 02:55PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (41)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 12 November 2013 11:59:55AM 1 point [-]

Shot the laser differently. For example, only make the comet smaller and change it's orbit so that some arbitrary city of earth gets blown up with knock-of effects.

If the AI has a naive "save humans" utility function, I don't see how this advantages it.

Hide a message that is far more difficult to detect than P is able to, but which can be used to communicate with a much later AI.

That kind of trade is indeed a problem, but you'd need to have a dangerous "much later AI" in the first place, which is a very bad thing anyway...

Circumventing physics entirely using advanced decision theories. (counterfactual trade, etc.)

That's a general risk - I'll analyse that later, if this seems to work.

The AI loopholes to send a physical femtobot "through the output channel" and escapes entirely.

The output channel is indeed dangerous - it is not constrained through this method, and needs to be controlled in other ways.

Comment author: Strange7 13 November 2013 12:57:57AM 2 points [-]

If the AI has a naive "save humans" utility function, I don't see how this advantages it.

I've met people who can lucidly argue that nuking a particular city or small region would produce many benefits for humanity as a whole, including reduced risk of politically-motivated extinction events down the line.

Also... you're going to an awful lot of trouble, here, to calculate a firing solution for a beam of light to hit a non-accelerating object in space. Realistically, if we know where the comet is well enough to realize it's headed for Earth, aiming a laser at it with non-sapient hardware is almost trivial. Why not an NP-complete problem?

Comment author: [deleted] 16 November 2013 09:28:32PM 1 point [-]

Why would an intelligent agent do better at an NP-complete problem than an unintelligent algorithm?

Comment author: Stuart_Armstrong 13 November 2013 11:45:19AM 1 point [-]

The laser problem is an illustration, a proof of concept of a developing idea. If that is deemed to work, I'll see how general we can make it.

Comment author: Armok_GoB 13 November 2013 04:59:17AM 1 point [-]

Normally I wouldn't make a post this contentless but I just HAVE to commend the general rationalist virtue on how this response was handled. I have no further responses.