= 27d567bc2d48d9f40e9ccc20ea370b15)
Luke_A_Somers comments on Superintelligence 9: The orthogonality of intelligence and goals - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (78)
How would they interact such that it's not simply adding over them, and they don't end up being predictably Dutch-bookable?
In the same way people's minds do. They are inconsistent but will notice the setup very quickly and stop. (I don't find Dutch book arguments very convincing, really).
Seems like a layer of inefficiency to have to resist temptation to run in circles rather than just want to go uphill.
There are two issues:
(a) In what settings do you want an architecture like that, and
(b) Ethics dictate we don't just want to replace entities for the sake of efficiency even if they disagree. This leads to KILL ALL HUMANS. So, we might get an architecture like that due to how history played out. And then it's just a brute fact.
I am guessing (a) has to do with "robustness" (I am not prepared to mathematise what I mean yet, but I am thinking about it).
People that think about UDT/blackmail are thinking precisely about how to win in settings I am talking about.
Pick a side of this fence. Will AI resist running-in-circles trivially, or is its running in circles all that's saving us from KILL ALL HUMANS objectives like you say in part b?
If the latter, we are so utterly screwed.