timtyler comments on Open Thread: February 2010, part 2 - Less Wrong

10 Post author: CronoDAS 16 February 2010 08:29AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (857)

You are viewing a single comment's thread. Show more comments above.

Comment author: timtyler 18 February 2010 08:53:33PM *  2 points [-]

No reply. Just so you know, the collective position on testing here is bizarre.

How you can think that superintelligent agents are often dangerous AND that a good way of dealing with this is to release an untested one on the world is really quite puzzling.

Hardly anyone ever addresses the issue. When they do, it is by pointing to AI box experiments, which purport to show that a superintelligence can defeat a lesser intelligence, even if well strapped down.

That seems irrelevant to me. To build a jail, for the smartest agent in the world, you do not use vastly less powerful agents as guards, you use slightly less powerful ones. If necesssary, you can dope the restrained agent up a bit. There are in fact all manner of approaches to this problem - I recommend thinking about them some more before discarding the whole idea of testing superintelligences.

Comment author: ciphergoth 18 February 2010 10:15:40PM 0 points [-]

No-one's against testing, that precaution should be taken, but it's not the most pressing concern at this stage.

Comment author: timtyler 18 February 2010 11:13:34PM *  1 point [-]

See "All of the above working first time, without testing the entire superintelligence", upthread. This is not the first time.