shminux comments on 'Dumb' AI observes and manipulates controllers - Less Wrong

33 Post author: Stuart_Armstrong 13 January 2015 01:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (19)

You are viewing a single comment's thread.

Comment author: shminux 13 January 2015 07:47:02PM 1 point [-]

I assume that you are alluding to spontaneously agentizing tools, a discussion initially triggered by the famous Karnofsky's post. In your example the feedback loop is closed, which is probably enough for a tool eventually gaining agency.

Comment author: Gram_Stone 16 January 2015 02:51:52PM 0 points [-]

Can you link to the post you're talking about? I've been thinking about this problem, although I had never read or thought the words 'spontaneously agentizing tools.'

Comment author: shminux 16 January 2015 04:15:07PM 1 point [-]

See Objection 2 in http://lesswrong.com/lw/cbs/thoughts_on_the_singularity_institute_si/ and related posts and comments.