= 27d567bc2d48d9f40e9ccc20ea370b15)
ata comments on Open Thread: May 2010 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (543)
Has anyone read The Integral Trees by Larry Niven? Something I always wonder about people supporting cryonics is why do they assume that the future will be a good place to live in? Why do they assume they will have any rights? Or do they figure that if they are revived, FAI has most likely come to pass?
Can't speak for any other cryonics advocates, but I find that to be likely. I see AI either destroying or saving the world once it's invented, if we haven't destroyed ourselves some other way first, and one of those could easily happen before the world has a chance to turn dystopian. But in any case, if you wake up and find yourself in a world that you couldn't possibly bear to live in, you can just kill yourself and be no worse off than if you hadn't tried cryonics in the first place.
Unless it's unFriendly AI that revives you and tortures you forever.
"unFriendly" doesn't mean "evil", just "not explicitly Friendly". Assuming you already have an AI capable of recursive self-improvement, it's easy to give it a goal system that will result in the world being destroyed (not because it hates us, but because it can think of better things to do with all this matter), but creating one that's actually evil or that hates humans (or has some other reason that torturing us would make sense in its goal system) would probably be nearly as hard as the problem of Friendliness itself, as gregconen pointed out.
Strongly unFriendly AI (the kind that tortures you eternally, rather than kills you and uses your matter to make paperclips) would be about as difficult to create as Friendly AI. And since few people would try to create one, I don't think it's a likely future.
Actually, it's quite possible to deny physical means of suicide to prisoners, and sufficiently good longevity tech could make torture for a very long time possible.
I think something like that (say, for actions which are not currently considered to be crimes) to be possible, considering the observable cruelty of some fraction of the human race, but not very likely-- on the other hand, I don't know how to begin to quantify how unlikely it is.