= 27d567bc2d48d9f40e9ccc20ea370b15)
IlyaShpitser comments on Superintelligence 9: The orthogonality of intelligence and goals - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (78)
There are two issues:
(a) In what settings do you want an architecture like that, and
(b) Ethics dictate we don't just want to replace entities for the sake of efficiency even if they disagree. This leads to KILL ALL HUMANS. So, we might get an architecture like that due to how history played out. And then it's just a brute fact.
I am guessing (a) has to do with "robustness" (I am not prepared to mathematise what I mean yet, but I am thinking about it).
People that think about UDT/blackmail are thinking precisely about how to win in settings I am talking about.
Pick a side of this fence. Will AI resist running-in-circles trivially, or is its running in circles all that's saving us from KILL ALL HUMANS objectives like you say in part b?
If the latter, we are so utterly screwed.