= 27d567bc2d48d9f40e9ccc20ea370b15)
Pentashagon comments on Friendly AI and the limits of computational epistemology - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (146)
There's a possibility that we are disastrously wrong about our own philosophical conclusions. Consciousness itself may be ethically monstrous in a truly rational moral framework. Especially when you contrast the desire for immortality with the heat death. What is the utility of 3^^^3 people facing an eventual certain death versus even just 2^^^2 or a few trillion?
I don't think there's a high probability that consciousness itself will turn out to be the ultimate evil but it's at least a possibility. A more subtle problem may be that allowing consciousness to exist in this universe is evil. It may be far more ethical to only allow consciousness inside simulations with no defined end and just run them as long as possible with the inhabitants blissfully unaware of their eventual eternal pause. They won't cease to exist so much as wait for some random universe to exist that just happens to encode their next valid state...
You could say the same of anyone who has ever died, for some sense of "valid" ... This, and similar waterfall-type arguments lead me to suspect that we haven't satisfactorily defined what it means for something to "happen."
It depends on the natural laws the person lived under. The next "valid" state of a dead person is decomposition. I don't find the waterfall argument compelling because the information necessary to specify the mappings is more complex than the computed function itself.