= 27d567bc2d48d9f40e9ccc20ea370b15)
Armok_GoB comments on Reduced impact AI: no back channels - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (41)
Ways for the AI to do other stuff:
Shot the laser differently. For example, only make the comet smaller and change it's orbit so that some arbitrary city of earth gets blown up with knock-of effects.
Hide a message that is far more difficult to detect than P is able to, but which can be used to communicate with a much later AI.
Circumventing physics entirely using advanced decision theories. (counterfactual trade, etc.)
Things that can't be mentioned here because they are cognitive hazards.
The AI loopholes to send a physical femtobot "through the output channel" and escapes entirely.
If the AI has a naive "save humans" utility function, I don't see how this advantages it.
That kind of trade is indeed a problem, but you'd need to have a dangerous "much later AI" in the first place, which is a very bad thing anyway...
That's a general risk - I'll analyse that later, if this seems to work.
The output channel is indeed dangerous - it is not constrained through this method, and needs to be controlled in other ways.
I've met people who can lucidly argue that nuking a particular city or small region would produce many benefits for humanity as a whole, including reduced risk of politically-motivated extinction events down the line.
Also... you're going to an awful lot of trouble, here, to calculate a firing solution for a beam of light to hit a non-accelerating object in space. Realistically, if we know where the comet is well enough to realize it's headed for Earth, aiming a laser at it with non-sapient hardware is almost trivial. Why not an NP-complete problem?
Why would an intelligent agent do better at an NP-complete problem than an unintelligent algorithm?
The laser problem is an illustration, a proof of concept of a developing idea. If that is deemed to work, I'll see how general we can make it.
Normally I wouldn't make a post this contentless but I just HAVE to commend the general rationalist virtue on how this response was handled. I have no further responses.
Is this a cognitive hazard you came up with yourself, or a "standard" one?
Both, but the obvious one in the context of this site was the one I had in mind the most.