MugaSofer comments on Reply to Holden on 'Tool AI' - Less Wrong

94 Post author: Eliezer_Yudkowsky 12 June 2012 06:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (348)

You are viewing a single comment's thread. Show more comments above.

Comment author: MugaSofer 01 May 2013 11:35:41PM *  -2 points [-]

So perhaps the supposed difference is that this AI won't actually take direct actions, merely make verbal suggestions. Well it's awful optimistic to suppose no one will get lazy or exigencies won't drive them to connect a simple script up to the machine which takes say sentences of the form "I recommend you deploy your troops in this manner." and directly sends the orders. Even if so the machine still takes direct action in the form of making statements that influence human behavior.

An Oracle determines which action would produce higher utility, then outputs it. An "Agent AGI" determines which output will produce higher utility, then outputs it. It's a question of optimizing the output or merely outputting optimization.

And yes, you can easily turn an Oracle into an Agent.