MugaSofer comments on Reply to Holden on 'Tool AI' - Less Wrong

94 Post author: Eliezer_Yudkowsky 12 June 2012 06:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (348)

You are viewing a single comment's thread. Show more comments above.

Comment author: MugaSofer 01 May 2013 11:40:56PM -1 points [-]

Regarding Eliezer's point #3 - I think this largely comes down to how strong one finds the argument for "tool A.I." I agree that one shouldn't expect SI to respond to every possible critique of its plans. But I think it's reasonable to expect it to anticipate and respond to the stronger possible critiques.

If, as you say, "Tool" AI is different to "Oracle" AI, you are the first person to suggest it AFAICT. Regardless of it's strength, it appears to be very difficult to invent; it seems unreasonable to expect someone to anticipate an argument when their detractors have also universally failed to do so (apart from you.)