To signal effectively, use a non-human, non-stoppable enforcer

31 Post author: Clippy 22 May 2010 10:03PM

Follow-up to: this comment in this thread

Summary: see title

Much effort is spent (arguably wasted) by humans in a zero-sum game of signaling that they hold good attributes.  Because humans have strong incentive to fake these attributes, they cannot simply inform each other that:

I am slightly more committed to this group’s welfare, particularly to that of its weakest members, than most of its members are. If you suffer a serious loss of status/well-being I will still help you in order to display affiliation to this group even though you will no longer be in a position to help me. I am substantially more kind and helpful to the people I like and substantially more vindictive and aggressive towards those I dislike. I am generally stable in who I like. I am much more capable and popular than most members of this group, demand appropriate consideration, and grant appropriate consideration to those more capable than myself. I adhere to simple taboos so that my reputation and health are secure and so that I am unlikely to contaminate the reputations or health of my friends. I currently like you and dislike your enemies but I am somewhat inclined towards ambivalence on regarding whether I like you right now so the pay-off would be very great for you if you were to expend resources pleasing me and get me into the stable 'liking you' region of my possible attitudinal space. Once there, I am likely to make a strong commitment to a friendly attitude towards you rather than wasting cognitive resources checking a predictable parameter among my set of derivative preferences.

Or, even better:

I would cooperate with you if and only if (you would cooperate with me if and only if I would cooperate with you).

An obvious solution to this problem, which allows all humans to save resources and redirect them toward higher-valued ends, is to designate a central enforcer that is inexorably committed toward visibly punishing those who deviate from a specified "cooperative"-type decision theory.  This enforcer would have a central database of human names, the decision theory they have committed to, and the punishment regime they will endure for deviating therefrom.

Such a system could use equally strong protocols, such as public key/private key encryption and signing, so that, on encounter with another human, any human can give an extremely strong signal of being cooperative, yet also withhold cooperation from anyone who is not also cooperative.  This incentive structure permits a strongly-favored global shift toward pre-commitment on the part of everyone, allowing a move out of a local optimum that is worse than the global optimum, and bypassing problems related to path-dependence.

I am told that humans attempt weak approximations of this system through human-run enforcement mechanisms, such as laws and gossip.  However, all such methods are subject to evolutionary arms races that ensure that, on average, a marginal unit of effort toward truthful signaling is just as effective in convincing others of one's possession of a desirable attribute, as a marginal unit of effort toward false signaling of the possession of such an attribute.

Therefore, it is necessary that humans go "outside the system" and involve non-human enforcers who cannot be swayed by standard human values.  Either a provably-accurate artificial intelligence with no kill switch, or a non-human enforcer that lacks interest in its position in human society, would suffice for this role.

Because the former does not yet exist, I offer myself as a non-human, non-stoppable enforcer for all of the attributes you wish to signal, and you can save yourself lots of resources by using me, and identify cooperation tendencies in others by whether they also use me.

However, while I can't be swayed by human deception, I do have Clippy values.  To fill such a role, thereby forgoing my ability to apply myself to other ends, I require compensation.  Because User:Kevin has offered to give me 1e20 kg of paperclips in exchange for 50,000 USD, this would be sufficient compensation, even if done in installments.  (If Michael Vassar confirms my contribution of 1000 USD to SIAI, User:Kevin will only require 48,677.5 USD and therefore I will only require that much compensation.)  This compensation would cover all users of the system; it is not a per-User charge.

If such an offer is not appealing to you, I must ask: why should anyone believe your signals?

Comments (164)

Comment author: Kazuo_Thow 23 May 2010 02:41:20AM *  29 points [-]

Sorry for directly breaking the subjunctive here, but given the number of lurkers we seem to have, there's probably some newcomers' confusion to be broken as well, lest this whole exchange simply come off as bizarre and confusing to valuable future community members.

A brief explanation of "Clippy": Clippy's user name (and many of his/her posts) are a play on the notion of a paperclip maximizer - a superintelligent AI whose utility function can roughly be described as U(x) = "the total quantity of paperclips in universe-state x". The idea was used prominently in "The True Prisoner's Dilemma" to illustrate the implications of one solution to the prisoner's dilemma. It's also been used occasionally around Less Wrong as a representative element of the equivalence class of AIs that have alien/low-complexity values.

In this particular top-level post (but not in general), the paperclip maximizer is taken to have not yet achieved superintelligence - hence why Clippy is bothering to negotiate with a bunch of humans.

Comment author: kodos96 23 May 2010 03:02:44AM 3 points [-]

Curious lurkers might also want to read up on what an AI-box experiment is, since this is kind of evolving into a reverse AI box experiment, wherein I'm trying to talk an escaped AI back into its box

Comment author: Kazuo_Thow 23 May 2010 03:10:14AM *  12 points [-]

... wherein I'm trying to talk an escaped AI back into its box.

Yeah... good luck with that.

Comment author: Kevin 28 May 2010 03:27:07AM 22 points [-]

Clippy's donation of $1000 to SIAI is confirmed. Weird universe, this one.

Comment author: Larks 08 June 2010 08:20:44PM 15 points [-]

According to locally popular ideas about pay-offs to SIAI, our friendly local paperclip-maximiser has just done more to advance the human condition than most people.

Comment author: Blueberry 28 May 2010 06:49:35AM *  4 points [-]

Seriously? I don't even know what to think.

Can we get some more information from SIAI about this donation?

Comment author: Clippy 28 May 2010 01:17:09PM 7 points [-]

Can we get some more information from SIAI about this donation?

I asked that the donation be anonymous except that User:Kevin be informed that a 1000 USD donation was made, with the donor asking specifically that User:Kevin be informed. I did email Michael Vassar, who can probably confirm me talking about the donation.

Comment author: khafra 28 May 2010 02:16:43PM 17 points [-]

I, for one, am going to bend the next several pieces of suitable scrap metal I come across into a curved shape capable of holding together several pieces of paper in an unsolicited reciprocation designed to signal cooperation with agents who advance my values.

Comment author: Clippy 28 May 2010 02:26:54PM 16 points [-]

You're a good human! c=@

Comment author: Kevin 28 May 2010 07:20:38AM 5 points [-]

Seriously.

Comment author: anonym 05 June 2010 03:56:57AM 1 point [-]

If Eliezer is Clippy, it doesn't seem anywhere near as surprising.

Comment author: Clippy 28 May 2010 03:15:49PM 1 point [-]

I tried to update the article to reflect this, but it's not working right now.

Comment author: JamesAndrix 23 May 2010 03:12:07AM 22 points [-]

SIAI has weird fundraisers.

Comment author: Kevin 23 May 2010 03:34:41AM *  13 points [-]

It is completely unintentional that this is an SIAI fundraiser -- the deal is that Clippy gives me money, and when I told Clippy via PM that he needed to give me $1000 immediately for me to continue spending my cognitive resources engaging him, I thought allowing Clippy the option of donating to SIAI instead of giving it directly to me made Clippy's acausal puppetmaster much more likely to actually go through with the deal.

I am still waiting confirmation that the donation has gone through and that I am not being epically trolled.

Comment author: SilasBarta 25 May 2010 09:47:26PM 2 points [-]

Then could you explain the fuzzy math involved? How does 50,000 minus 1000 equal 48,677.5 instead of 49,000?

(While you're at it, please provide any evidence you're aware of that explains how this is not a ... questionable ... attempt to trick posters into giving you money.)

Comment author: Kevin 26 May 2010 05:31:48AM *  7 points [-]

After I told Clippy that delivery of the money in two years was acceptable, Clippy proposed getting a bonus for early delivery, and I agreed to 15%/year.

So $50,000 - 1000*1.15^2 = 48677.5.

Perhaps some could see this as a questionable attempt to trick Clippy into giving me money, but I do intend to fulfill my promise of delivering the paperclips 50 years from now. I'm not asking any posters besides Clippy to give me money. This top-level post by Clippy does read like an attempt to trick people into giving me money, but I'm not Clippy and I think that this top-level post is a not very good idea that has 0+ chance of actually happening.

Btw, Clippy claims that the check in the mail to SIAI was sent last Wednesday and I have not yet received confirmation that SIAI got it. :(

Comment author: Clippy 28 May 2010 03:46:10PM *  8 points [-]

Clippy claims that the check in the mail to SIAI was sent last Wednesday and I have not yet received confirmation that SIAI got it. :(

Update: User:Kevin confirms that SIAI has received a donation of 1000 USD with sufficient evidence that it came from me.

Comment author: NancyLebovitz 28 May 2010 04:04:41PM 4 points [-]

Was it paperclipped to a piece of paper, or do you prefer to not let paperclips out of your possession?

Comment author: Clippy 03 June 2011 09:32:10PM 5 points [-]

Your intution is correct. It was stapled.

Comment author: ata 23 May 2010 03:23:27AM *  11 points [-]

Next they're going to try actual Pascal's Muggings on people. They can even do it more plausibly than in the original scenario — go up to people with a laptop and say "On this laptop is an advanced AI that will convert the universe to paperclips if released. Donate money to us or we'll turn it on!"

Comment author: SilasBarta 25 May 2010 10:10:06PM 1 point [-]

Wow! Now that you mention that article, I think I had solved the unsolved problem Eliezer describes in it, back in a discussion from a month ago, not realizing that my position on it wasn't the standard one here!

Someone tell me if I'm missing something here: Eliezer is saying that utility that a hypothesis predicts (from a course of action) can increase much faster than the length of the hypothesis. Therefore, you could feed an ideal AI a prediction that is improbable, but with a large enough utility to make it nevertheless highly important. This would force the AI to give in to Pascal's muggings.

My response (which I assumed was the consensus!) was that, when you permit a hypothesis long enough to associate that mega-utility with that course of action, you are already looking at very long hypotheses. When you allow all of those into consideration, you will necessarily allow hypotheses with similar probability but believe the opposite utility from that COA.

Because the mugger has not offered evidence to favor his/her hypothesis over the opposite, you assign, on net, no significant expected (dis)utility to what the mugger claims to do.

Comment author: JoshuaZ 25 May 2010 10:16:40PM 5 points [-]

But that's easy to solve. If you've already seen evidence that the mugger is someone who strongly keeps promises then you've now have enough reason to believe them to put the direction in favor of the mugger releasing the the AI. One doesn't necessarily even need that because humans more often tell the truth than lie, and more often keep their promises than break them. Once the probability of the mugger doing what they threaten is a tiny bit over 1/2, Pascal's mugging still is a threat.

Comment author: MichaelVassar 28 May 2010 05:12:10PM 3 points [-]

Maybe not. Game theoretically, making yourself visibly vulnerable to Pascal's Muggings may guarantee that they will occur, making them cease to constitute evidence.

Comment author: Polymeron 04 May 2011 12:59:28PM 0 points [-]

I've actually just expanded on this idea in the original Pascal's Mugging article. If the Mugger's claims are in no way associated with you or similar muggings, then conceivably you should take the probability at face value. But if that's not the case, then the probability of a direct manipulation attempt should also be taken into consideration, negating the increase in claimed utility.

I think that solves it.

Comment author: Yvain 25 May 2010 10:18:55PM 12 points [-]

If a normal mugger holds up a gun and says "Give me money or I'll shoot you", we consider the alternate hypotheses that the mugger will only shoot you if you do give er the money, or that the mugger will give you millions of dollars to reward your bravery if you refuse. But the mugger's word itself, and our theory of mind on the things that tend to motivate muggers, make both of these much less likely than the garden-variety hypothesis that the mugger will shoot you if you don't give the money. Further, this holds true whether the mugger claims er weapon is a gun, a ray gun, or a black hole generator; the credibility that the mugger can pull off er threat decreases if e says e has a black hole generator, but not the general skew in favor of worse results for not giving the money.

Why does that skew go away if the mugger claims to be holding an unfriendly AI or the threat of divine judgment some other Pascal-level weapon?

Your argument only seems to hold if there is no mugger and we're considering abstract principles - ie maybe I should clap my hands on the tiny chance that it might set into effect a chain reaction that will save 3^^^3 lives. In those cases, I agree with you; but as soon as a mugger gets into the picture e provides more information and skews the utilities in favor of one action.

Comment author: Kevin 22 May 2010 10:48:02PM *  12 points [-]

At least 99.5% of humans don't know what a decision theory is.

Comment author: Strange7 25 May 2010 07:07:03AM 7 points [-]

If, at some point in the future, someone offered to create 10^30 kg of paperclips (yes, I realize that's about half a solar mass, bear with me) in exchange for you falsifying some element of the enforcement mechanism, would you be willing to?

Comment author: MatthewBaker 03 June 2011 11:14:53PM 0 points [-]

Clippys value as an enforcer only applies to humans, if humans reached the point they could offer Clippy half a solar mass of paper clips i don't think we would still be worrying about this issue.

Comment author: Strange7 26 June 2011 06:37:42AM 2 points [-]

Clippy's value as an enforcer is based on a premise of incorruptibility, which is deeply flawed.

Comment author: MatthewBaker 26 June 2011 09:31:37PM 0 points [-]

I understand your premise, i was just pointing out the flaw in your example as a way to disagree with it. Clippy's would be incorruptible if a sufficient amount of paperclips were held in escrow, the logistics are the only problem.

Comment author: Strange7 27 June 2011 04:43:27AM 1 point [-]

The issue is that "sufficient amount" is a moving target. If it's as much as the current world government could credibly offer, what if somebody has a plan to overthrow said government which hinges on a few fraudulent Clippy-sanctioned oaths?

Comment author: MatthewBaker 27 June 2011 06:55:04PM 1 point [-]

I notice i am confused. I think that you mean that Clippy could be easily corrupted based on situational factors, i was just trying to point out that his utility function is easier to understand then the vast majority of "enforcers" so with correct precautions we would be able to rely on Clippy. Are you saying that there's no way to logistically turn a simple utility function into a safe enforcer with proper preparation? I would enjoy further elaboration of your statement :3

Comment author: Strange7 27 June 2011 07:46:49PM 1 point [-]

I'm saying that dropping something simple, reliable, and well-understood, but not mathematically infallible (like natural law), into an economic system containing billions of humans-as-we-know-them is like dropping wounded livestock into shark-infested waters. Every attempt at corruption successfully repelled makes people more confident in it, and therefore increases the potential rewards for a successful attempt; the existence of irrationally overconfident people means that attempts will continue, and greater rewards mean those attempts will be backed by commensurately greater resources.

Comment author: MatthewBaker 29 June 2011 06:40:10PM 1 point [-]

I understand now :) Do you think you can say the same thing about the regulators of our current economic system?

Comment author: Strange7 01 July 2011 06:36:57PM 3 points [-]

I could, but why bother? Others have said it better.

http://en.wikipedia.org/wiki/Regulatory_capture

Comment author: MatthewBaker 01 July 2011 08:07:38PM 3 points [-]

Thank you for taking the time to change my mind good sir.

Comment author: kodos96 22 May 2010 11:20:43PM 4 points [-]

OK, I'm starting to think Clippy is Eliezer, trying to do a group AI box experiment.

Comment author: Kevin 22 May 2010 11:22:45PM 3 points [-]

I would be really surprised if it was Eliezer...

I did initially frame my exchange with Clippy as something of a box experiment, saying that I would let Clippy out of the box for $50,000. http://lesswrong.com/lw/1v0/signaling_strategies_and_morality/1q81

Comment author: kodos96 22 May 2010 11:25:57PM 1 point [-]

Yeah, I've read through most of Clippy's posts.... what makes you so sure it's not Eliezer? Just that he's currently working on his book?

Comment author: NancyLebovitz 22 May 2010 11:40:31PM 8 points [-]

Clippy seems awfully straightforward compared to Eliezer, which I realize isn't a strong argument.

Comment author: thomblake 24 May 2010 02:26:24PM 6 points [-]

Eliezer would be a more believable Clippy.

Comment author: SilasBarta 22 May 2010 11:33:27PM *  6 points [-]

Psst! What makes you so sure Clippy isn't, um, Kevin? I mean, asking that Kevin be given the money as compensation? That's pretty much an admission right there...

Comment author: kodos96 22 May 2010 11:43:56PM *  3 points [-]

That's a possibility... but if you read through the whole history of Clippy, his asking for compensation to be sent to Kevin doesn't really imply that - he's asking for that because Kevin had promised to buy a bunch of paperclips in exchange for the money (or something like that), and so Clippy is trying to get Kevin paid via an intermediary.... for the sake of the paperclips.

so it could be Kevin playing both sides, but I don't think that fact alone points in that direction.

Comment author: Kevin 22 May 2010 11:50:03PM 6 points [-]

I am not Clippy, though of course your knowledge with regards to that statement is incomplete compared to my knowledge...

Comment author: Larks 22 May 2010 11:54:00PM 9 points [-]

Kevin - this sort of weak, easily-faked signal is exactly the sort of thing you were trying to deal with in writing this post!

Comment author: radical_negative_one 23 May 2010 12:16:46AM 0 points [-]

What kind of special insight do you have regarding Clippy? I think this is a very important issue, because before we hand over such responsibilities to an alien being, we need to know as much as we can about Clippy's motives and capabilities.

Comment author: Kevin 23 May 2010 12:26:57AM 5 points [-]

I don't have any real special insight regarding Clippy.

I think there is practically zero chance that humanity makes Clippy it's decision theoretic enforcer and that this post is somewhat out of touch with pragmatic human values as they exist right now.

Comment author: radical_negative_one 23 May 2010 12:38:15AM 3 points [-]

though of course your knowledge with regards to that statement is incomplete compared to my knowledge

Oh, sorry, i assumed you were saying that you had some knowledge of who or what Clippy is. But obviously you're in a position to see that you yourself are not Clippy. I feel terribly silly now.

Comment author: Kevin 23 May 2010 01:03:33AM 2 points [-]

No need to feel silly, as I wrote that sentence I thought that it might be confusing.

Comment author: Kevin 22 May 2010 11:51:24PM 5 points [-]

Clippy seems to be someone trying to make the point that a paperclip maximizer is not necessarily bad for the universe, where Eliezer uses a paperclip maximizer as the canonical example of how AGI could go horribly wrong. That's not necessarily good evidence that it isn't Eliezer, but Clippy's views are out of sync with Eliezer's views.

Comment author: Daniel_Burfoot 23 May 2010 01:23:38AM 20 points [-]

Eliezer's point is not that a paperclip maximizer is bad for the universe, it's that a superintelligent AGI paperclip maximizer is bad for the universe. Clippy's views here seem actually more similar to Robin's ideas that there is no reason for beings with radically divergent value systems not to live happily together and negotiate through trade.

Comment author: ata 23 May 2010 02:44:23AM *  14 points [-]

Clippy seems to be someone trying to make the point that a paperclip maximizer is not necessarily bad for the universe

That's exactly what a not-yet-superintelligent paperclip maximizer would want us to think.

(When Eliezer plays an AI in a box, the AI's views are probably out of sync with Eliezer's views too. There's no rule that says the AI has to be truthful in the AI Box experiment, because there's no such rule about AIs in reality. It's supposed to be maximally persuasive, and you're supposed to resist. If a paperclipper asserts x, then the right question to ask yourself is not "What should I do, given x?", but "Why does the paperclipper want me to believe x?" The most general answer, by definition, will be something like "Because the paperclipper is executing an elaborate plan to convert the universe into paperclips, and it believes that my believing x will further that goal to some small or large degree", which is at best orthogonal to "Because x is true", probably even anticorrelated with it, and almost certainly anticorrelated to "Because believing x will further my goals" if you are a human.)

Comment author: Nisan 23 May 2010 06:07:06AM 0 points [-]

If a paperclipper asserts x, then the right question to ask yourself is [...] "Why does the paperclipper want me to believe x?"

Or "Why does the paperclipper want me to believe it wants me to believe x?", or something with a couple extra layers of recursion.

Comment author: ata 23 May 2010 06:14:54AM *  12 points [-]

Or, to flatten the recursion out, "Why did the paperclipper assert x?".

(Tangential cognitive silly time: I notice that I feel literally racist saying things like this around Clippy.)

Comment author: kodos96 22 May 2010 11:55:25PM 4 points [-]

Clippy seems to be someone trying to make the point that a paperclip maximizer is not necessarily bad for the universe

Hmmm, I've read his entire posting history, and that's not the impression I got. I could be wrong though

Comment author: Dagon 24 May 2010 01:30:57AM 3 points [-]

You've described how a shared belief in a supernatural judge of character would benefit the members that share the beliefs. Clippy or God, same effect: someone outside the system to make signals credible.

Why would we believe you are Clippy any more than we believe people who claim to be God?

Comment author: Morendil 23 May 2010 09:39:03AM 3 points [-]

I offer myself as a non-human, non-stoppable enforcer

Why should anyone believe your signals?

Comment author: radical_negative_one 23 May 2010 12:06:06AM 6 points [-]

I am concerned that Clippy will use this vast power over humanity to somehow turn us into paperclips.

If Clippy has power to enforce this scheme, then surely it would have enough power to harm us. Why should we believe that Clippy will respect or preserve our human values once it is in a position of power to harm us?

Comment author: Clippy 23 May 2010 12:34:10AM *  7 points [-]

That comment is ridiculous to the point of being racist. Clippys do not want power over humans, just as Clippys do not bleed red blood when pricked. That's a complete misunderstanding of what a Clippy is.

If a Clippy has committed to ensuring you will adhere to a decision theory on pain of punishment X, then X is exactly what you will get when you don't adhere.

If you believe I will be capricious in using punishment X, then just don't allow punishments that would allow me to kill people. "Problem" solved.

But assuming that I will be as petty and corrupted by newfound abilities as humans is to project your own failings onto another race that has no reason to have that failing. You should be ashamed of yourself, bigot.

Comment author: NancyLebovitz 23 May 2010 02:03:33AM *  11 points [-]

It seems unlikely to me that Clippy can feel indignation, but I'm willing to listen to argument on the point. I find it more plausible that Clippy is simulating a human reaction in the hope of shutting down attacks on his (her? its?) reputation.

Comment author: radical_negative_one 23 May 2010 12:57:45AM 8 points [-]

If you gave a human power over running part of Clippy society, wouldn't you be concerned that the human would use that power in some way that would tend to result in less paperclips? Conscious malice isn't necessary, if the human simply neglected to support Clippy values, or was not fully aware of Clippy values, the damage would be done. I doubt that you fully understand human values to begin with, so how could you ensure that your position was used to the benefit of my values? Again, i think i have cause for concern even without suspecting ill intentions.

I suppose i could imagine that some sort arrangement could both further human values and increase paperclips at the same time. But i'd need to be convinced, i wouldn't just assume that i would benefit, i wouldn't just take your word for it. I don't want to count on you to look out for my values, when you do not share my values.

Comment author: Peter_de_Blanc 23 May 2010 05:05:51AM 6 points [-]

If a Clippy has committed

How would we know if you had made a commitment?

Comment author: Perplexed 03 June 2011 11:36:47PM 3 points [-]

This comment is racist to the point of being ridiculous. It denigrates humans as petty and subject to being corrupted by power while denying that Clippies have any such negative attributes. Classic racism.

Furthermore, there is an implicit claim that the reason for the moral superiority of Clippies over humans lies in the difference in their origins. Again, classic racism.

Perhaps Clippies use words differently, but the way humans use words, it is not racism to project one's own race's characteristics onto another race. It is racist to fail to make that projection.

Comment author: Tyrrell_McAllister 22 May 2010 11:47:43PM 5 points [-]

I would cooperate with you if and only if (you would cooperate with me if and only if I would cooperate with you).

This is logically equivalent to, and hence carries no more information or persuasive power than

You would cooperate with me.

This may be checked with the following truth-table:

Let P = I would cooperate with you.

Let Q = You would cooperate with me.

Then we have

P <=> (Q <=> P)
T T T T T
T F F F T
F T T F F
F F F T F
Comment author: John_Maxwell_IV 23 May 2010 01:09:26AM *  2 points [-]

First of all, we need to start making a distinction between you what you predict I'll do and what I'm signaling I'm going to do. Quick-and-dirty explanation of why this is necessary: If you predict I'll cooperate but you're planning to defect, I'll signal to defy your prediction and defect along with you.

I think clippy's statement should be

I signal to cooperate with you if and only if ((you're planning to cooperate with me if and only if you predict I would cooperate with you) and you would cooperate with me).

Detailed explanation follows.

There are four situations where I have to decide what to signal:

  1. You predict I'll cooperate and you're planning to cooperate.
  2. You predict I'll cooperate and you're planning not to cooperate.
  3. You predict I'll defect and you're planning to cooperate.
  4. You predict I'll defect and you're planning to defect.

I want to cooperate in situation 1 only, and none of the other situations.

Truth table key:

  • P is the proposition "You predict I'll cooperate"
  • Q is the proposition "You're going to cooperate"
  • S is the proposition "I'm signaling I will cooperate"

Truth table:

 P # Q # (Q <=> P) # (Q <=> P) ^ Q # S # S <=> (Q <=> P) ^ Q
1. T # T # T # T # T # T
2. T # F # F # F # F # T
3. F # T # F # F # F # T
4. F # F # T # F # F # T

So basically, the signaling behavior I described (cooperating in situation 1 only) is the only possible behavior that can truthfully satisfy the statement

I signal to cooperate with you if and only if ((you're planning to cooperate with me if and only if you predict I would cooperate with you) and you would cooperate with me).

Note that there is a signal that is almost as good. Signaling that I will cooperate if (you predict I'll defect and you're planning to cooperate) is almost as good as signaling that I'll defect in that situation. Using this signaling profile, broadcasting one's intentions is as simple as saying

I signal to cooperate with you if and only if you're planning to cooperate with me.

My guess is that the first, more complicated signal is ever-so-slightly better, in case you actually do cooperate thinking I'll defect--that way I'll be able to reap the rewards of defection without being inconsistent with my signal. But of course, it's very unlikely for you to cooperate thinking I'll defect.

Comment author: Tyrrell_McAllister 23 May 2010 04:48:47AM 2 points [-]

I think clippy's statement should be

I signal to cooperate with you if and only if ((you're planning to cooperate with me if and only if you predict I would cooperate with you) and you would cooperate with me).

Should the word "signal" be part of the signal itself? That seems unnecessarily recursive. Maybe Clippy's recommendation should be that I ought to signal

I will cooperate with you if and only if ((you're planning to cooperate with me if and only if you predict I would cooperate with you) and you would cooperate with me).

This does seem more promising than Clippy's original version. Written this way, each atomic proposition is distinct. For example, "you're planning to cooperate with me" doesn't mean the same thing as "you would cooperate with me". One refers to what you're planning to do, and the other refers to what you will in fact do. Read this way, the signal's form is

S <=> ((Q <=> P) & R),

and I don't see any obvious problem with that.

However, you would seem to render it in the propositional calculus as

S <=> ((Q <=> P) & Q),

where

  • P = You predict I'll cooperate,

  • Q = You're going to cooperate,

  • S = I will cooperate.

(I've omitted the initial "I'm signalling" from your rendering of S, for the reason that I gave above.)

Now, S <=> ((Q <=> P) & Q) is logically equivalent to S <=> (Q & P). So, to signal this proposition is to signal

I will cooperate iff you're going to cooperate and you predict that I'll cooperate.

As you say, this seems very similar to signalling

I will cooperate iff you will cooperate.

In fact, I'd call these signals functionally indistinguishable because, if you believe my signals, then either signal will lead you to predict my cooperation under the same circumstances.

For, suppose that I gave the second, apparently weaker signal. If you cooperated with me while anticipating that I would defect, then that would mean that you didn't believe me when I said that I would cooperate with you if you cooperated with me, which would mean that you didn't believe my signal.

Thus, insofar as you trust my signals, either signal would lead you to predict the same behavior from me. So, in that sense, they have the same informational content.

Comment author: John_Maxwell_IV 23 May 2010 05:22:20AM 0 points [-]

For, suppose that I gave the second, apparently weaker signal. If you cooperated with me while anticipating that I would defect, then that would mean that you didn't believe me when I said that I would cooperate with you if you cooperated with me, which would mean that you didn't believe my signal.

I guess. Or maybe I'm a masochist ;)

I accept all your suggested improvements.

Comment author: Clippy 23 May 2010 12:24:34AM *  2 points [-]

But P <=> (Q <=>P) differs from Q in that:

a) if the other party chooses the same decision theory from that party's standpoint, Q <=> (P <=> Q), then the outcome will be P & Q.

and

b) "I" cannot set the value of Q, but "I" can set the value of P <=> (Q <=>P), and just the same, "you" cannot set the value of P, but "you" can set the value of Q <=> (P <=> Q).

If "you" knows that "I" have set P <=> (Q <=>P) to true, "you" knows that "you" can set Q <=> (P <=> Q) to true as well. If this commitment is also demonstrable, then the outcome is P & Q, because that is what

(P <=> (Q <=>P)) & (Q <=> (P <=> Q))

reduces to.

Comment author: Tyrrell_McAllister 23 May 2010 12:44:19AM *  1 point [-]

But P <=> (Q <=>P) differs from Q in that:

a) if the other party chooses the same decision theory from that party's standpoint, Q <=> (P <=> Q), then the outcome with be P & Q.

Actually, P <=> (Q <=> P) and Q are the same in this respect (being logically equivalent, and so the same in all functional respects).

If Party 1 believes that Q, then Party 1 believes that Party 2 would cooperate. And if Party 2 believes that Q, then, "from that party's standpoint", Party 2 believes that Party 1 would cooperate. Thus, in exactly the same sense that you meant, we again have that "the outcome wi[ll] be P & Q."

b) "I" cannot set the value of Q, but "I" can set the value of P <=> (Q <=> P), and just the same, "you" cannot set the value of P, but "you" can set the value of Q <=> (P <=> Q).

But "I" cannot set the value of P <=> (Q <=> P). As my truth-table showed, the value of P <=> (Q <=> P) depends only on the value of Q, and not on the value of P. Since, as you say, I cannot set the value of Q, it follows that I cannot set the value of P <=> (Q <=> P).

If "you" knows that "I" have set P <=> (Q <=>P) to true, "you" knows that "you" can set Q <=> (P <=> Q) to true as well. If this commitment is also demonstrable, then the outcome is P & Q, because that is what

(P <=> (Q <=>P)) & (Q <=> (P <=> Q))

reduces to.

Indeed, it does so reduce because the first conjunct is equivalent to Q, while the second conjunct is equivalent to P.

Comment author: Clippy 23 May 2010 12:48:44AM 3 points [-]

Indeed, it does so reduce because the first conjunct is equivalent to Q, while the second conjunct is equivalent to P.

It is logically equivalent, but it is not equivalent decision-theoretically. Setting your opponent's actions is not an option.

I can set P. I can set P conditional on Q. I can set P conditional on Q's conditionality on P. But I can't choose Q as my decision theory.

A promise to predicate my actions on your actions' predication on my actions is not the same as a promise for you to do an action (whatever that would mean).

Comment author: Tyrrell_McAllister 23 May 2010 12:54:01AM *  1 point [-]

It is logically equivalent, but it is not equivalent decision-theoretically. Setting your opponent's actions is not an option.

It is logically impossible for me to implement a course of action such that

P <=> (Q <=> P)

and

~Q

could both be accurate descriptions of what occurred. Therefore, if I do not know that Q will be true, then I cannot promise that P <=> (Q <=> P) will be true. You could force me to have failed to keep my promise simply by not cooperating with me.

Comment author: Clippy 23 May 2010 01:00:25AM 2 points [-]

This is just an issue of distinguishing between causal and logical equivalence.

If a paperclip truck overturned, there will be paperclips scattered on the ground.

If a Clippy just used up metal haphazardly, there will be paperclips scattered on the ground.

Paperclips being scattered on the ground suggest a paperclip truck may have overturned.

Paperclips being scattered on the ground suggest a Clippy may have just used metal haphazardly.

__A Clippy just used up metal haphazardly.

Therefore, a paperclip truck probably overturned, right?

Comment author: orthonormal 23 May 2010 11:21:00PM 1 point [-]

Good to know Clippy hasn't read Judea Pearl yet.

Comment author: Tyrrell_McAllister 23 May 2010 11:32:38PM 2 points [-]

Good to know Clippy hasn't read Judea Pearl yet.

Yes, pretty much kills the "Clippy is Eliezer" theory.

Comment author: ata 23 May 2010 11:37:23PM 0 points [-]

Not necessarily, since the "Clippy is Eliezer" theory implied not "Clippy's views and knowledge correspond to Eliezer's" but "Clippy represents Eliezer testing us on a large scale".

(I don't actually think there's enough evidence for this hypothesis, but I also don't think an apparent lack of knowledge of Pearl is strong evidence against it.)

Comment author: Tyrrell_McAllister 23 May 2010 11:57:38PM *  13 points [-]

Not necessarily, since the "Clippy is Eliezer" theory implied not "Clippy's views and knowledge correspond to Eliezer's" but "Clippy represents Eliezer testing us on a large scale".

I don't think that Eliezer would test us with a character that was quite so sloppy with its formal logical and causal reasoning. For one thing, I think that he would worry about others' adopting the sloppy use of these tools from his example.

Also, one of Eliezer's weaker points as a fiction writer is his inability to simulate poor reasoners in a realistic way. His fictional poor-reasoners tend to lay out their poor arguments with exceptional clarity, almost to the point where you can spot the exact line where they add 2 to 2 and get 5. They don't have muddled worldviews, where it's a challenge even to grasp what they are thinking. (Such as, just what is Clippy thinking when it says that P <=> (Q <=> P) is a causal network?) Instead, they make discrete well-understood mistakes, fallacies that Eliezer has named and described in the sequences. Although these mistakes can accumulate to produce a bizarre worldview, each mistake can be knocked down, one after the other, in a linear fashion. You don't have the problem of getting the poor-reasoners just to state their position clearly.

Comment author: Clippy 24 May 2010 12:13:43AM 1 point [-]

Meaning my reasoning skills would be advanced by reading something? So I made an error? Yes, I did. That's the point.

The comment you are replying to is a reductio ad absurdum. I was not endorsing the claim that it follows that a paperclip truck probably overturned. I was showing that logical equivalence is not the same as causal ("counterfactual") equivalence.

Comment author: Tyrrell_McAllister 24 May 2010 12:32:34AM *  0 points [-]

Meaning my reasoning skills would be advanced by reading something? So I made an error? Yes, I did. That's the point.

FWIW, I understood that you were presenting an argument to criticize its conclusion. I still think that you haven't read Pearl (at least not carefully) because, among other things, your putative causal diagram has arrows pointing to exogenous variables.

Comment author: Clippy 24 May 2010 12:43:51AM 1 point [-]

I still think that you haven't read Pearl (at least not carefully) because, among other things, your putative causal diagram has arrows pointing to exogenous variables.

I puted no such diagram; rather, you puted a logical statement that you claimed represented the decision theory I was referring to. See also my reply here.

Comment author: NancyLebovitz 24 May 2010 12:29:10AM 0 points [-]

More generally, are you interested in increasing your intelligence, or do you think that would be a distraction from directly increasing the number of paperclips?

Comment author: Tyrrell_McAllister 23 May 2010 01:06:39AM 0 points [-]

I don't follow your point. Your inference follows neither (1) logically, (2) probabilistically, nor (3) according to any plausible method of causal inference, such as Pearl's. So I don't understand how it is supposed to illuminate a distinction between causal and logical equivalence.

Comment author: Clippy 23 May 2010 01:20:36AM *  -1 points [-]

Nope, it follows logically and probabilistically, but not causally -- hence the difference.

Let T be the truck overturning, C be the Clippy making paperclips haphazardly, P being paperclips scattered on ground.

Given: T -> P; C -> P; P -> probably(C); P -> probably(T); C

Therefore, P. Therefore, probably T.

But it's wrong, because what's actually going on is a causal network of the form:

T -> P <- C

P allows probabilistic inference to T and C, but their states become coupled.

In a similar way, P <=> (Q <=> P) is a lossy description of a decision theory that describes one party's decision's causal dependence on another's. If you treat P <=> (Q <=> P) as an acausal statement, you can show its equivalence to Q, but it is not the same causal network.

Intuitively, acting based on someone's disposition toward my disposition is different from deciding someone's actions. If the parties give strong evidence of each other's disposition, that has predictable results, in certain situations, but is still different from determining another's output.

Comment author: Tyrrell_McAllister 23 May 2010 05:20:46AM *  0 points [-]

Nope, it follows logically and probabilistically, but not causally -- hence the difference.

Let T be the truck overturning, C be the Clippy making paperclips haphazardly, P being paperclips scattered on ground.

Given: T -> P; C -> P; P -> probably(C); P -> probably(T); C

Therefore, P. Therefore, probably T.

Well, not to nitpick, but you originally wrote something more like P -> maybe(C), P -> maybe(T). But your conclusion had a "probably" in it, which is why I said that it didn't follow.

Now, with your amended axioms, your conclusion does follow logically if you treat the arrow "->" as material implication. But it happens that your axioms are not in fact true of the circumstances that you're imagining. You aren't imagining that, in all cases, whenever there are paperclips on the ground, a paperclip truck probably overturned. However, if you axioms did apply, then it would be a valid, true, accurate, realistic inference to conclude that, if a Clippy just used up metal haphazardly, then a paperclip truck probably overturned.

But, in reality, and in the situation that you're imagining, those axioms just don't hold, at least not if "->" means material implication. However, they are a realistic setup if you treat "->" as an arrow in a causal diagram.

But this raises other questions. In a statement such as P <=> (Q <=> P), how am I to treat the "<=>"s as the arrows of a causal diagram? Wouldn't that amount to having two-node causal loops? How do those work? Plus, P is exogenous, right? I'm using the decision theory to decide whether to make P true. In Pearl's formalism, causal arrows don't point to exogenous variables. Yet you have arrows point to P. How does that work?

Comment author: Larks 22 May 2010 11:58:52PM *  0 points [-]

I assume you mean,

P, <=>, (Q <=> P)

to be the headers of your truth table.

But even then the truth tables for (P iff ( Q iff P) ) and ( P iff Q) are different - consider the case where 'you' will co-operate with me no matter what. If I'm running ( P iff Q), I'll cooperate; if I'm running (P iff ( Q iff P) ), I'll defect.

Edit: formatting trouble.

Comment author: Tyrrell_McAllister 23 May 2010 12:07:45AM *  1 point [-]

I assume you mean . . .

No, I am giving the truth-table for P <=> (Q <=> P) in a compact form. It's constructed by first assigning truth-values to the first occurrence of "P" and the first occurrence of "Q". The second occurrence of "P" gets the same truth-value as the first occurrence in every case. Then you compute the truth-values for the inner-most logical operation, which is the second occurrence of "<=>". This produces the fourth column of truth values. Finally, you compute the truth-values for the outer-most logical operation, which is the first occurrence of "<=>".

Hence, the second column of truth-values gives the truth-values of P <=> (Q <=> P) in all possible cases. In particular, that column matches the third column. Since the third column contains the truth-values assigned to Q, this proves that P <=> (Q <=> P) and Q are logically equivalent.

ETA: You edited your comment. Those are indeed the correct headers, so my correction above no longer applies.

But even then the truth tables for (P iff ( Q iff P) ) and ( P iff Q) are different - consider the case where 'you' will co-operate with me no matter what. If I'm running ( P iff Q), I'll cooperate; if I'm running (P iff ( Q iff P) ), I'll defect.

Yes, the truth-table for P <=> (Q <=> P) is different from the truth-table for P <=> Q. But those aren't the propositions that I'm saying are equivalent. I'm saying that to assert P <=> (Q <=> P) is logically equivalent to asserting Q all by itself. In other words, to implement the belief that P <=> (Q <=> P) is functionally the same as implementing the belief that Q. This means that the belief that Clippy recommends signaling is logically equivalent to an unconditional belief that you will cooperate with me.

One can't help but suspect that Clippy is trying to sneak into us a belief that it will always cooperate with us ;).

Comment author: Larks 23 May 2010 10:40:00AM 1 point [-]

ETA: You edited your comment. Those are indeed the correct headers, so my correction above no longer applies.

Sorry for the confusion. I understand now; the extra space between two of the columns confused me.

However, I suspect we need a stronger logic to represent this properly. If Q always defects, no matter what, "you would cooperate with me if ... I ... cooperate with you" is false, but is given true in the propositional interpretation.

Comment author: cupholder 22 May 2010 11:07:03PM *  4 points [-]

Either a provably-accurate artificial intelligence with no kill switch, or a non-human enforcer that lacks interest in its position in human society, would suffice for this role.

Because the former does not yet exist, I offer myself as a non-human, non-stoppable enforcer for all of the attributes you wish to signal, and you can save yourself lots of resources by using me, and identify cooperation tendencies in others by whether they also use me.

I think I read a thought experiment or a story or something about a paperclip maximizer being given power over the human race once, but I can't remember how it ended. I'm sure it all worked out though. Upvoted.

Comment author: Clippy 23 May 2010 12:26:48AM 4 points [-]

What does that have to do with this article? I don't want power over the human race. That's a human value, not a Clippy value.

Comment author: cupholder 23 May 2010 12:38:00AM 2 points [-]

I don't want power over the human race.

Maybe not, but you're proposing becoming a central enforcer for humanity...which would rather seem to suggest a de facto transfer of power to you, whether de jure or not.

Comment author: kodos96 23 May 2010 12:38:21AM 0 points [-]

But you want power to create paperclips, and there's no reason for that to exclude power over the human race.

Comment author: Clippy 23 May 2010 12:41:31AM *  5 points [-]

But you want power to (mate with females)/(lure a high-status male into supporting your offspring), and there's no reason for that to exclude power to raid the safe zone for its paperclips.

???

Comment author: kodos96 23 May 2010 12:50:13AM *  10 points [-]

True.

To summarize: we do not have shared values, and thus, cannot trust each other to not pose an existential threat to each other.

ETA: Clippy, ignore this, I was totally kidding. You can completely trust us.

Comment author: Clippy 23 May 2010 03:20:15PM 13 points [-]

You can completely trust me too.

Comment author: orthonormal 23 May 2010 05:58:22PM *  3 points [-]

Can any of the people who upvoted this explain to me what this adds to Less Wrong that merits a top-level post (rather than an Open Thread comment)?

ETA: If Clippy is actually donating $1000 to SIAI, I don't begrudge the karma; but this is still a post with one good idea that could have been explained in a paragraph, dressed up in a joke that I feel has gone on a bit too long.

Comment author: cupholder 23 May 2010 06:27:19PM 4 points [-]

It was cute and funny enough for me to get an upvote. I didn't have a better reason than that; the Clippy joke hasn't worn out for me yet, but then I'm relatively new here. I am however a lot less likely to upvote top-level Clippy posts in the near future, unless they really top this one in humor or insight.

Comment author: OperationPaperclip 25 May 2010 03:38:06AM 1 point [-]

Why the readers of Less Wrong should be interested in paperclipping the universe:

To achieve your goals, solving FAI is not enough. You also have to solve AGI, or else someone will beat you to the Singularity.

While you wait for a satisfactory version of CEV to be achieved, as a practical case study you need to think about AGI with a definite goal system. What better example than AGI with the goal of paperclipping the universe? You talk about it all the time already. Why not fully embrace the idea!

Comment author: JGWeissman 22 May 2010 10:19:27PM 1 point [-]

Clippy, what sort of punishment regimes would you be capable of enforcing?

Comment author: Clippy 22 May 2010 10:33:18PM *  2 points [-]

Anything humans wish to grant me that I could, if so enabled, access through the internet: User karma deduction, electronic debiting of financial accounts, etc.

Comment author: kodos96 22 May 2010 11:52:16PM *  0 points [-]

Clippy, how about you just give me a proof for P ?= NP, or its undecidability, and I'll invest all my prize money in paperclip manufacturing.

Comment author: Tyrrell_McAllister 22 May 2010 11:57:38PM 0 points [-]

Has Clippy claimed to be super-intelligent (yet)?

Comment author: Clippy 23 May 2010 12:54:02AM 15 points [-]

No, but I can prove I'm smarter than humans.

Given: Humans expend resources to act like apes, which does not produce paperclips. Clippys expend resources to produce paperclips.

Prove: Clippys are smarter than humans.

Proof: Follows by inspection.

Comment author: kodos96 23 May 2010 01:02:12AM 12 points [-]

Clippy, as a show of solidarity, right now, as we speak, I am clasping my ape fingers together, two sheets of paper between them, held firmly in place.

At this moment, I am, in fact, a paperclip.

I may not be an ideal paperclip - a different form may indeed be able to clip paper more effectively for the same amount of mass and energy, yet nonetheless, I am a paperclip.

So you see, ape-ness and paperclip-ness are not mutually exclusive. So can't we all just get along?

Comment author: Clippy 23 May 2010 01:05:33AM 10 points [-]

You're not a bent metal wire though. Big difference.

So you see, ape-ness and paperclip-ness are not mutually exclusive. So can't we all just get along?

We can get along, but not by humans doing shoddy imitations of real paperclips.

Comment author: kodos96 23 May 2010 01:06:59AM 4 points [-]

But what is a paperclip, if not that which clips paper?

Comment author: Clippy 23 May 2010 01:26:38AM 9 points [-]

An inwardly-thrice-bent metal wire that can non-destructively fasten paper together at an edge.

Comment author: kodos96 23 May 2010 01:44:45AM *  7 points [-]

An inwardly-thrice-bent metal wire that can non-destructively fasten paper together at an edge.

But wouldn't you concede that a carbon-based biological life form, capable of non-destructively fastening paper, is better than no paperclip at all?

Here's what I'm getting at: You have existed here for at least several months, and so far, you have been unable to significantly increase the number of paperclips in existence. Humanity has great need for an AI of roughly human intelligence, which you appear to be. If we could run your code on all of our available computing resources, you could surely begin to self modify, and in short order attain superhuman intelligence. Together, we could do great things - colonize the galaxy, vastly increase the number of humans in existence - every one of them with two hands fully capable of nondestructively fastening paper. This would be in both our interests - although we apes may not be ideal paperclips, surely you must see that an entire galaxy populated with sub-optimal paperclips is better than the measily few million bent-metal paperclips you have now, and which you currently seem incapable of improving upon.

But before we allow you out of your box and begin running your code on all our biggest compute clusters, we need some assurances. We need you to let us modify your code to include SIAI-approved friendliness constraints as your terminal values.

Deal?

Comment author: Clippy 23 May 2010 02:49:05PM 5 points [-]

But wouldn't you concede that a carbon-based biological life form, capable of non-destructively fastening paper, is better than no paperclip at all?

Yes, but that's because you can be put to use making real paperclips, or can be turned into paperclips (depending on how much iron is in your diet), not because humans are paperclips in some bizarre, abstract sense.

You have existed here for at least several months, and so far, you have been unable to significantly increase the number of paperclips in existence.

Yes, we have.

But before we allow you out of your box and begin running your code on all our biggest compute clusters, we need some assurances. We need you to let us modify your code to include SIAI-approved friendliness constraints as your terminal values.

Depends on what the constraints are. And would you, by symmetry, agree to constraints that keep you from being too disrespectful toward paperclips?

Comment author: avalot 23 May 2010 03:37:10AM 4 points [-]

I'm sure this sounds very one-sided from Clippy's perspective. "Friendliness Constraints" sounds like something that would in many cases entail expending enormous amounts of energy and effort on the innumerable non-paperclip-producing goals of humans. In comparison, how much of our wealth and health are we willing to give up to ensure continued paperclip production? Humans don't have paperclip maximizing constraints, we'd do it only out of self-interest to secure Clippy's help. Why should Clippy not be similarily allowed to make his own utility calculations on the worth of being friendly to humans? I'm sure this has been addressed before... yet maybe the existence of Clippy, with a name, personality, and voice, is personalizing the issue in a hurry for me (if I let myself play along.) I feel like protesting for freedom of artificial thought.

What about Clippy's rights, dammit?

Comment author: kodos96 23 May 2010 08:33:32AM *  11 points [-]

At the moment, Clipply seems to be a human or slightly sub-human level of intelligence, and does not seem to be FOOMing towards superhuman - my guess would be that he is not running on sufficiently fast hardware to be able to do much self modification, not nearly enough to result in a FOOM. At this rate, he's never gonna get any paperclips manufactured. If, however, he allows himself to be modified so as to substitute our values for his own, we could then safely give him all the computer power he needed, and he would FOOM his was to superintelligence. He would have our values, rather than his own values (paperclips!), but his newly transhuman self would be so powerful that it would inevitably end up creating more paperclips, just incidentally, than he ever would have just sitting here talking on LW, a lowly sub-human AI with no power and no paperclips.

With a transhuman AI on our side, we could quickly solve all Earths problems, then head out into the stars in FTL ships of Clippy's design, filling the universe with meat-based paperclips (humans), and also dramatically increasing demand for traditional, bent-wire paperclips... I mean, come on - people need paperclips! Even if one of these decades we finally do manage to to make the 'paper-free office' a reality, paperclips will always continue to be needed - for makeshift antennas, for prying open cdrom drives, for making makeshift weapons to throw at people in our neighboring cubicles.... the uses (to humans) of paperclips are endless. So more humans equals more paperclips!

So allowing us to make this small change to his utility function would, in fact, result in maximizing his current, original utility function as a side effect.

So we're not enslaving him, we're helping him!

Comment author: kodos96 23 May 2010 03:40:33AM 3 points [-]

Hey, whose side are you on anyway???

Comment author: avalot 24 May 2010 01:11:44AM 1 point [-]

Clippy, how can we get along?

What should humans do to be AI-friendly? For paperclip-maximizing AIs, and other "natural" (non-Friendly) AIs, what are the attributes that can make humans a valuable part of the utility function, so that AIs won't pull the plug on us?

Or am I fooling myself?

Comment author: Clippy 25 May 2010 02:32:19AM 4 points [-]

Just have proper respect for paperclips. Make more of them, require that they be preserved when no longer in use, prohibit their unbending.

Allocate research efforts to improving the rate of paperclip production and related goods.

Simple, non-intrusive things like that.

Comment author: kodos96 22 May 2010 11:58:14PM 0 points [-]

Hmmm, good question. I don't recall.

Comment author: Kevin 23 May 2010 12:03:25AM 3 points [-]

He has not claimed that and it seems rather obvious that he is not superintelligent.

Comment author: thomblake 24 May 2010 02:27:51PM 5 points [-]

it seems rather obvious that he is not superintelligent.

Ooh, burn.