= 27d567bc2d48d9f40e9ccc20ea370b15)
cousin_it comments on I've had it with those dark rumours about our culture rigorously suppressing opinions - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
Comments (857)
Alternative which I view as being more frightening:
For any given human, its CEV involves that human winning at zero-sum, possibly even negative-sum, games (status would be one of these). As such, the best way to maximize the current collection of humanity's CEV would be to create new agents to which current humans defeat in zero-sum games.
That is, for every current human, create a host of new agents (all of whom are quite human for all intents and purposes) of whom the current human is emperor.
Note: if this is the case, I doubt pseudo-agents will suffice. Just as humans do not wish to love pseudo-humans (that is, humans who cannot really love), humans do not wish to win zero-sum games against pseudo-humans (that is, humans who cannot really lose zero-sum games, with all that losing these games entails).
CEV will probably have many contributions from people who don't want the AI to create almost-human slaves. Do you think such desires will lose out in reflective equiibrium?
I never said they would be slaves, although I certainly did imply it. I probably should not have said 'emperor.' A more appropriate term would have been something like 'grand-champion' or 'big winner.'
I certainly hope not. However, I personally have no idea.