private_messaging comments on I think I've found the source of what's been bugging me about "Friendly AI" - Less Wrong

8 Post author: ChrisHallquist 10 June 2012 02:06PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (31)

You are viewing a single comment's thread. Show more comments above.

Comment author: private_messaging 12 June 2012 07:03:27AM *  -1 points [-]

That sounds way less scary when you consider actual software that is approaching recursive self improvement and get more specific than vague "increase in power". It's just generic ignorant anti-technology talk that relies on vague concepts like "power" and dissipates once you get in any way specific.

The software also tends not to do what you want it to do for sake of this argument. There's an enormous gap between 'not doing exactly what we want' and doing exactly what you want for this argument to work. The automated engineering software simulates microscopic material interaction; vague self improvement and increases in power only make it better at not doing unrelated stuff.