= 27d567bc2d48d9f40e9ccc20ea370b15)
Armok_GoB comments on Wanted: backup plans for "seed AI turns out to be easy" - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (62)
I'm inclined towards the view that we shouldn't even try to capture all human complexity of value. Instead, we should just build a simple utility function that captures some value that we consider important, and sacrifices everything else. If humans end up unhappy with this, the AI is allowed to modify us so that we become happy with it.
Yes, being turned to orgasmium is in a sense much worse than having an AI satisfying all the fun theory criteria. But surely it's still much better than just getting wiped out, and it should be considerably easier to program than something like CEV.
I think it sounds worse. If an AI more friendly than that turns out to be impossible I'd probably go for the negative utilitarian route and give the AI a goal of minimizing anything that might have any kind of subjective experience. Including itself once it's done.