= 27d567bc2d48d9f40e9ccc20ea370b15)
lessdazed comments on Guardian Angels: Discrete Extrapolated Volitions - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (9)
A more convenient possible world was alluded to when I asked about excluding some individuals.
No merging?
Maybe, but I had also asked about the relative difficulty of calculating CEV and DEV. If DEV is easier, perhaps possible rather than impossible, that's an advantage of it.
War is a risk, it includes the possibility of mutual destruction, particularly if offense is more powerful. You don't think they'd merge resources and values instead of risking it?
Most likely scenario I agree, still less than probable,
Cyberwar is different than regular war in that all competently performed attacks are inherently anonymous. Attacks performed very competently are also undetectable. This is very destabilizing. And it gets worse; while AIs might try to get around this by all merging together, none of them would be able to prove they hadn't hidden a copy of themselves somewhere.
I don't think undetectability solves things. Offensive subsystems could survive their creator's demise like two people in a grenade lobbing fight.
Suppose all hid a copy, the merged AI would still be more powerful than any hidden copies, and if it was destroyed everyone would be a small copy again. If there were many AIs, an individual would be banking on its ability to defeat a much larger entity. Offense is more powerful on most scales and technological levels but not by incomprehensible orders of magnitude.