You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

torekp comments on Superintelligence 16: Tool AIs - Less Wrong Discussion

7 Post author: KatjaGrace 30 December 2014 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (36)
Tags: superintelligence-reading-group futurism ai

Comments (36)

You are viewing a single comment's thread. Show more comments above.

Comment author: torekp 10 January 2015 01:24:21AM 1 point [-]

Please explain more, especially regarding Katja's point 5

  1. One obvious problem with tools is that they maintain humans as a component in all goal-directed behavior. If humans are some combination of slow and rare compared to artificial intelligence, there may be strong pressure to automate all aspects of decisionmaking, i.e. use agents.

Is there any solution better than attempting to make and enforce a treaty against moving from tool to agent?

Comment author: [deleted] 10 January 2015 12:23:50PM 1 point [-]

First of all, those two assumptions of humans being slow and rare compared to artificial intelligence are dubious. Humans are slow at some things, but fast at others. If the architecture of the AGI differs substantially from the way humans think, it is very likely that the AGI would not be very fast at doing some things humans find easy. And early human-level AGIs are likely to consume vast supercomputing resources; they're not going to be cheap and plentiful.

But beyond that, the time frame for using tool AI may be very short, e.g. on the order of 10 years or so. There isn't a danger of long-term instability here.

Powered by Reddit Powered by Reddit