You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

David_Gerard comments on Tools want to become agents - Less Wrong Discussion

12 Post author: Stuart_Armstrong 04 July 2014 10:12AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (81)
Tags: self-modification modification ai agents tools

Comments (81)

You are viewing a single comment's thread. Show more comments above.

Comment author: David_Gerard 04 July 2014 01:32:59PM *  -1 points [-]

It would, however, be interesting to. This discussion has come around before. What I said there:

We may need another word for "agent with intentionality" - the way the word "agent" is conventionally used is closer to "daemon", i.e. tool set to run without user intervention.

I'm not sure even having a world-model is a relevant distinction - I fully expect sysadmin tools to be designed to form something that could reasonably be called a world model within my working lifetime (which means I'd be amazed if they don't exist now). A moderately complex Puppet-run system can already be a bit spooky.

Note that mere daemon-level tools exist that many already consider unFriendly, e.g. high-frequency trading systems.

Comment author: mwengler 04 July 2014 03:58:54PM *  3 points [-]

Note that mere daemon-level tools exist that many already consider unFriendly, e.g. high-frequency trading systems.

A high-frequency trading system seems no more complex or agenty to me than rigging a shotgun to shoot at a door when someone opens the door from the outside. Am I wrong about this?

To be clear, what I think I know about high-frequency trading systems is that through technology they are able to front run certain orders they see to other exchanges when these orders are being sent to multiple exchanges in a non-simultaneous way. The thing that makes them unfriendly is that they are designed by people who understand order dynamics at the microsecond level to exploit people who trade lots of stock but don't understand the technicalities of order dynamics. That market makers are allowed to profit by selling information flow to high-frequency traders that, on examination, allows them to subvert the stated goals of a "fair" market is all part of the unfriendliness.

But high-frequency programs execute pretty simple instructions quite repeatably, they are not adaptive in a general sense or even particularly complex, they are mostly just fast.

Comment author: David_Gerard 04 July 2014 04:27:55PM *  0 points [-]

Mmm ... I think we're arguing definitions of ill-defined categories at this point. Sort of "it's not an AI if I understand it." I was using it as an example of a "daemon" in the computing sense, a tool trusted to run without further human intervention - not something agenty.

Comment author: TheAncientGeek 05 July 2014 01:12:30PM *  0 points [-]

Intentionality meaning " "the power of minds to be about, to represent, or to stand for, things, properties and states of affairs", ...or intentionally meaning purpose?

Powered by Reddit Powered by Reddit