Thomas comments on Q&A with experts on risks from AI #1 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (66)
This need not be the case. Whenever we talk about software "wanting" something, we are of course speaking metaphorically. It might be straightforward to build a super-duper Watson or Wolfram Alpha, that responds to natural queries "intelligently", without the slightest propensity to self-modify or radically alter the world. You might even imagine such a system having a background thread trying to pre-compute answers to interesting questions and share them with humans, once per day, without any ability to self-modify or significant probability of radical alteration to human society.
The one who poses the Answering machine is the friendly or is not friendly. The whole system - Oracle+Owner(User) - is a rouge or quite friendly SAI then.
The whole problem shifts a little, but doesn't change very much for the rest of the humanity.