= 27d567bc2d48d9f40e9ccc20ea370b15)
paper-machine comments on Help, help, I'm being oppressed! - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (141)
That's because it seems more likely that there's only one FAI to rule them all, and whatever values it has will dominate the light-cone.
A FAI is more likley to actually be a FAI if people don't engage in a last desperate war for ownership of all the universe for eternity at the time of its construction.
The current proposed solution to avoid such negative sum arms race (where aggressive action and recklessness reduce the likelihood of a friendly AI for nearly all other human value sets, but increases the likelihood of one for your particular value set) has been to hope that our values aren't really different, we're just (for now) too dumb to see this.
It's a bit worse than that. The "hope" seems to be more along the lines of:
Nevermind how a nascent valueless AI is supposed to convince itself to go back into the box.