= 27d567bc2d48d9f40e9ccc20ea370b15)
MugaSofer comments on Failed Utopia #4-2 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (247)
I don't believe in trying to make utopias but in the interest of rounding out your failed utopia series how about giving a scenario against this wish.
I wish that the future will turn out in such a way that I do not regret making this wish. Where I is the entity standing here right now, informed about the many different aspects of the future, in parallel if need be (i.e if I am not capable of groking it fully then many versions of me would be focused on different parts, in order to understand each sub part).
I'm reminded by this story that while we may share large parts of psychology, what makes a mate have an attractive personality is not something universal. I found the cat girl very annoying.
Fixed that for you.
Haven't you ever played the corrupt-a-wish game?
Wish granted: horror as the genie/ai runs a matrix with copy after copy of you, brute forcing the granting of possible wishes, most of which turn out to be an absolute disaster. But you aren't allowed to know that happens, because the AI goes..."insane" is the best word I can think of, but it's not quite corrrect...trying to grant what is nearly an ungrantable wish, freezing the population into stasis untill it runs out of negentropy and crashes...
Now that's not to say friendly AI can't be done, but it WON'T be EASY.
If your wish isn't human-proof, it probably isn't AI-safe.
Yes, I have. Saying "the genie goes insane because it's not smart enough to grant your wish" is not how you play corrupt-a-wish. You're supposed to grant the wish, but with a twist so it's actually a bad thing.
perhaps I didn't make the whole "it goes and pauses the entire world while trying to grant your wish" part clear enough...
Trying and failing to grant the wish is not the same as granting it, but it's actually terrible.
If the AI can't figure out the (future) wishes of a single human being, then it is insufficiently intelligent, and thus not the AI you would want in the first place.
The genie vanishes, taking with it any memory that you ever met a genie. Since you would not remember making the wish, and since you would see no evidence of a wish having been made, you would not regret having made the wish.