= 27d567bc2d48d9f40e9ccc20ea370b15)
xxd comments on Wanted: backup plans for "seed AI turns out to be easy" - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (62)
I'm inclined towards the view that we shouldn't even try to capture all human complexity of value. Instead, we should just build a simple utility function that captures some value that we consider important, and sacrifices everything else. If humans end up unhappy with this, the AI is allowed to modify us so that we become happy with it.
Yes, being turned to orgasmium is in a sense much worse than having an AI satisfying all the fun theory criteria. But surely it's still much better than just getting wiped out, and it should be considerably easier to program than something like CEV.
"But surely it's still much better than just getting wiped out"
I think that is the key here. If "just getting wiped out" is the definition of unfriendly then "not gettting wiped out" should be the MINIMUM goal for a putative "friendly" AI.
i.e. "kill no humans".
It starts to get complex after that. For example: Is it OK to kill all humans, but freeze their dead bodies at the point of death and then resurrect one or more of them later? Is it OK to kill all humans by destructively scanning them and then running them as software inside simulations? What about killing all humans but keeping a facility of frozen embryos to be born at a later date?