timtyler comments on AI indifference through utility manipulation - Less Wrong

4 Post author: Stuart_Armstrong 02 September 2010 05:06PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (53)

You are viewing a single comment's thread. Show more comments above.

Comment author: timtyler 06 September 2010 05:05:12PM *  1 point [-]

They will necessarily be samples from our mindspace - otherwise they wouldn't be so useful.

Computers so far have been very different from us. That is partly because they have been built to compensate for our weaknesses - to be strong where we are weak. They compensate for our poor memories, our terrible arithmetic module, our poor long-distance communications skills - and our poor ability at serial tasks. That is how they have managed to find a foothold in society - before maastering nanotechnology.

IMO, we will probably be seeing a considerable amount more of that sort of thing.

Comment author: jacob_cannell 06 September 2010 05:32:05PM *  0 points [-]

Computers so far have been very different from us. [snip]

Agree with your point, but so far computers have been extensions of our minds and not minds in their own right. And perhaps that trend will continue long enough to delay AGI for a while.

For for AGI, for them to be minds, they will need to think and understand human language - and this is why I say they "will necessarily be samples from our mindspace".