= 27d567bc2d48d9f40e9ccc20ea370b15)
shminux comments on Tools versus agents - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (39)
It looks like you argue that a sufficiently powerful tool is instrumentally indistinguishable from an agent. Can you formulate it as a theorem and outline the steps required to prove it?
I don't think the theorem is true. I think there's some daylight between superintelligent tools and agents - just not much. Not nearly as much as, say, between an Oracle and a sovereign AI.
One thing I am not clear about is whether you are saying that a tool AI spontaneously develops what appears like intentionality or not. It sure seems that that is what you are saying, initially with a human in the feedback loop, until the suggestion to "create an AI with these motivations" is implemented. If so, then why are you saying that "there's some daylight between superintelligent tools and agents"?
Then formulate a weaker version of the theorem that you think is true.
Tool AIs and Agent AIs given the same task will not lead to the same outcomes, nor result in the same routes being taken. However, they are likely dangerous in broadly similar ways, and many warnings and precautions for one of these apply also to the other.