= 27d567bc2d48d9f40e9ccc20ea370b15)
Stuart_Armstrong comments on Reply to Holden on 'Tool AI' - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (348)
Minor point from Nick Bostrom: an agent AI may be safer than a tool AI, because if something goes unexpectedly wrong, then an agent with safe goals should turn out to be better than a non-agent whose behaviour would be unpredictable.
Also, an agent with safer goals than humans have (which is a high bar, but not nearly as high a bar as some alternatives) is safer than humans with equivalently powerful tools.
How is this helpful? This is true by definition of the word "safer". The problem is knowing whether an agent has safer goals, or what "safer" means.
I don't think this makes any sense. A tool AI has no autonomous behavior. It computes a function. Its output has no impact on the world until a human uses it. The phrase "tool AI" implies to me that we are not talking about an AI that you ask, for instance, to "fix the economy"; we are talking about an AI that you ask questions such as, "Find me data showing whether lowering taxes increases tax revenue."