You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Gunnar_Zarncke comments on Tackling the subagent problem: preliminary analysis - Less Wrong Discussion

5 Post author: Stuart_Armstrong 12 January 2016 12:26PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (16)

You are viewing a single comment's thread.

Comment author: Gunnar_Zarncke 13 January 2016 07:31:45AM 1 point [-]

and it's not clear that a sense of identity prevents the creation of subagents in the first place

It doesn't. Humans do create sub-agents all the time to do their bidding. No I do not mean children. I mean sending out other people do errands. Yes, this is imperfect, but the AIs sub agents wouldn't be perfect either. They may fail. In particular any sub-agent may fail and any restriction that calls to fail never (including sub-agent failure) is bound to cause malfunction is the first place.

Comment author: Stuart_Armstrong 13 January 2016 10:39:54AM 1 point [-]

There's some informal suggestions (which I don't think much of, so I didn't really go into deep analysis) that use a sense of identity as the basis of controlling subagents. I didn't want to go into the weeds of that in this post.

Comment author: Gunnar_Zarncke 13 January 2016 11:43:27AM 0 points [-]

Yes. Some notion of identity is needed in any case for the AI. it has to encompass its executive functions as least. Identity distinguishes the AI from what is not the AI. I see no reason why this couldn't include sub-agents. It is more a question of where the line is drawn not if. I'm looking forward to a future post of yours on identity.