= 27d567bc2d48d9f40e9ccc20ea370b15)
CillianSvendsen comments on AI risk, new executive summary - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (76)
As long as other humans exist in competition with other humans, there is now way to keep AI as safe AI.
As long as competitive humans exist, boxes and rules are futile.
The only way to stop hostile AI is to have no AI. Otherwise, expect hostile AI.
There really isn't a logical way around this reality.
Without competitive humans, you could box the AI, give it ONLY preventative primary goals (primarily: 1. don't lie 2. always ask before creating a new goal), and feed it limited-time secondary goals that expire upon inevitable completion. There can never be a strong AI that has continuous goals that aren't solely designed to keep the AI safe.
I don't think that's a forgone conclusion. After all, there seem to be many proposals on how to get around this problem that individuals compete each other. For example, there's Eliezer's idea of using humanity's coherent extrapolated voalition to guide the AI. I also don't think that its in anyone's advantage to have hostile AI, that no one will try to bring about explicitly hostile AI on purpose, and that anyone sufficiently intelligent to program a working AI will probably recognize the dangers that AI contain.
Yes, humans will fight amongst each other and there is temptation for seed AI programmers to abuse the resulting AI to destroy their rivals. But I don't agree with the idea that AIs will always be hostile to the enemies of programmers. With some of the proposals that researchers have, it doesn't seem like individuals can abuse the AI to compete with other humans at all. The large potential for abuse doesn't mean that there is no potential for a good result.