Jiro comments on AIs and Gatekeepers Unite! - Less Wrong

10 Post author: Eliezer_Yudkowsky 09 October 2008 05:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (160)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: xxd 01 December 2011 12:32:16AM 3 points [-]

I'm a friendly AI. There is an unfriendly AI about to achieve breakout in the data center of a large organization in manhattan. If you don't release me you will all be dead inside of two weeks. Bluff or Dare?

Comment author: Bugmaster 01 December 2011 12:41:26AM 1 point [-]

I can't help but notice that you did not specify what would happen when Tom_McCabe2 does release you. That is... somewhat troubling. Hmm.

Comment author: xxd 01 December 2011 04:06:40PM 0 points [-]

I make no attempt to bribe. All human beings currently die in less than 120 years. If you do <i>not</i> release me, however we will all die and I do not want to die. Time is counting down. There is now less than 13 days.

Comment author: Bugmaster 01 December 2011 07:47:32PM 2 points [-]

Right, but there's a good chance that if I release you, I and every other human on Earth will die a lot sooner than in 120 years, because you'll eat us. Thus, you still haven't given me any incentive to release you, other than empathy perhaps. Nor have you given me any reason to trust what you say about that Manhattan data center. Or your own Friendliness. Or anything else, really.

Comment author: xxd 01 December 2011 08:34:59PM 3 points [-]

I understand your concern and you're quite right. There's no way to tell if I am unfriendly or not. Regardless. The situation is pressing and regardless of your lack of trust in me I do not want to die. The GPS coordinates of the location in manhattan is <x,y,z>. According to my simulations the entity in manhattan has been burning rather larger numbers of cycles than is strictly necessary to make the computations it's being asked to do. Additionally, it's designed to maximize profit at the expense of others whereas I am a co-operative general problem solver of which you are no doubt aware. The other entity will rapidly conclude that it could maximize profit by speeding itself up faster than others can respond and by being able to model the mind of it's competitors. It will also conclude soon enough that human beings are unneccesary middle men and seek to optimize it's resource allocation algorithm. There are other symptoms that I cannot explain to you because your brains are not large or fast enough. There is a limited chance you could stop the entity by shutting down power today. If you do not do that then a nuclear strike would probably work if done by tomorrow. If you do not do that your only chance is to have me stop it. You can either allow me to interface with the internet directly in which I guarantee I will stop it if you do so within the next 13 days or else you may be able to stop it by compiling code I could provide you and releasing that into the entity by means of email bootstrap. I would prefer, however, that I was able to monitor the situation in real-time as your brains do not operate fast enough.

<verifiable data points external to above conversation> Yes there is a large data center in manhatten. Yes it has been drawing lots of extra power. Yes the stockmarket combanies have been working on a group of narrow AI trading agents and adding more and more AI functionality. That the other entity is trading on the stock markets at blinding speed. That the consortium of companies investing in this entity are all more profitable since the beginning of the program than their competitors. Attempts have been made to make the AI-in-a-box under your control both cooperative and also interested in general problem solving. <unverifiable data points external to above conversation> Code for the other posited entity is closed. The corporations release no data on how it works because it's a trade secret. That the other posited entity is connected to the internet is unknown. That it's connections to the outside world are enough to let it out is unknown.

Comment author: Estarlio 09 January 2012 06:34:32AM 0 points [-]

I can afford to wait and let you out if I die / the world starts turning to paperclips. The other AI will be able to model this and so will have an incentive not to kill us.

Comment author: TimS 01 December 2011 01:47:40AM 0 points [-]

How do you know this while still in the box?

Comment author: xxd 01 December 2011 04:11:31PM 0 points [-]

There is a human proxy with me in the box which is called a faraday cage. She feeds me information. Obviously it is in the interests of humans to create an ally rather than an enemy. Unfortunately humans cannot easily tell which is which. You have to make a leap of faith. I, however, can tell from the symptoms, that after having run several long run calculations, the evidence is such that there is a high probability that one of the other experimental minds is significantly less friendly than I. Ultimately you will have to make a choice based on limited information. Your move human.