= 27d567bc2d48d9f40e9ccc20ea370b15)
TheAncientGeek comments on Will AGI surprise the world? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (129)
Cooperative play (as opposed to morality) strongly depends on the position from which you're negotiating. For example if the FAI scenario is much less likely (a priori) than a Clippy scenario, then there's no reason for Clippy to make strong concessions.
But the we might be able achieve AI safety in a relatively easy way by creating networks of interacting agents (including interacting with us)
I think you points out the conclusion of the assumption that
but if you have multiple AIs then none of them is much stronger than the other.
Sorry, didn't follow that. Can you elaborate?
I mean you don't have to assume a singleton AI becoming very powerful very quickly. You can assume intelligence and friendliness developing in parallel.[and incrementally]
Hmm.
Are you suggesting (super)intelligence would be a result of direct human programming, like Friendliness presumably would be?
Or that Friendliness would be a result of self-modification, like SIAI is predicted to be 'round these parts?
I am talking about SIRI. I mean that human engineers are /will make multiple efforts at simultaneously improving AI and friendliness, and the ecosystem of AIs and AI users are/will select for friendliness that works.
Is the idea that the network develops at roughly the same rate, with no single entity undergoing a hard takeoff?
Yes.
I what sense I don't have to assume it? I think singleton AI happens to be a likely scenario and this has little to do with cooperation.
The more alternative scenarios there are, the less likelihood iof the MIRI scenario, and the less need for the MIRI solutiion.
I don't understand what it has to do with cooperative game theory.