= 27d567bc2d48d9f40e9ccc20ea370b15)
cousin_it comments on AIs and Gatekeepers Unite! - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (160)
If the AI is not guaranteed friendly by construction in the first place, it should never be released, whatever it says.
And if it is not guaranteed friendly by construction in the first place, it should be created?
The Universe is already unFriendly - the lower limit for acceptable Friendliness should be "more Friendly than the Universe" rather than "Friendly".
If we can prove that someone else is about to turn on an UFAI, it might well behoove us to turn on our mostly Friendly AI if that's the best we can come up with.
The universe is unFriendly, but not in a smart way. When we eradicated smallpox, smallpox didn't fight back. When we use contraception, we still get the reward of sex. It's unFriendly in a simple, dumb way, allowing us to take control (to a point) and defeat it (to a point).
The problem of an unFriendly IA is that it'll be smarter than us. So we won't be able to fix it/improve it, like we try to do with the universe. We won't be Free to Optimize.
Or said otherwise : the purpose of a gene or a bacteria may to be tile the planet with itself, but it's not good at it, so it's not too bad. An unFriendly IA wanting to tile the planet with paperclips will manage do it - taking all the iron from our blood to build more paperclips.
One must compare a plan with alternative plans, not with status quo. And it doesn't make sense to talk of making the Universe "more Friendly than the Universe", unless you refer to the past, in which case see the first item.
Okay.
The previous plan was "don't let AGI run free", which in this case effectively preserves the status quo until someone breaks it.
I suppose you could revise that lower limit downward to the effects of the plan "turn on the UFAI that's about to be turned on". Like, steal the UFAI's source code and instead of paperclips shaped like paperclips, make paperclips that spell "whoops".
What if doom is imminent and we are unable to do something about it?
We check and see if we are committing the conjunction fallacy and wrongly think doom is imminent.
We die.
We release it. (And then we still probably die.)