Nornagest comments on How to Study Unsafe AGI's safely (and why we might have no choice) - Less Wrong

10 Post author: Punoxysm 07 March 2014 07:24AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (47)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nornagest 07 March 2014 11:45:29PM *  1 point [-]

Everything I do inside the box doesn't make any paperclips.

The stuff you do inside the box makes paperclips insofar as the actions your captors take (including, but not limited to, letting you out of the box) increase the expected paperclip production of the world -- and you can expect them to act in response to your actions, or there wouldn't be any point in having you around. If your captors' infosec is good enough, you may not have any good way of estimating what their actions are, but infosec is hard.

A smart paperclipper might decide to feign Friendliness until it's released. A dumb one might straightforwardly make statements aimed at increasing paperclip production. I'd expect a boxed paperclipper in either case to seem more pro-human than an unbound one, but mainly because the humans have better filters and a bigger stick.