waitingforgodel comments on Open Thread: July 2010, Part 2 - Less Wrong

6 Post author: Alicorn 09 July 2010 06:54AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (770)

You are viewing a single comment's thread. Show more comments above.

Comment deleted 23 September 2010 08:26:57AM *  [-]
Comment author: Wei_Dai 23 September 2010 06:32:01PM *  7 points [-]

Since I assume he doesn't want to have existential risk increase, a credible threat is all that's necessary.

Perhaps you weren't aware, but Eliezer has stated that it's rational to not respond to threats of blackmail. See this comment.

(EDIT: I deleted the rest of this comment since it's redundant given what you've written elsewhere in this thread.)

Comment author: wedrifid 23 September 2010 07:41:56PM *  8 points [-]

This is true, and yes wfg did imply the threat.

(Now, analyzing not advocating and after upvoting the parent...)

I'll note that wfg was speculating about going ahead and doing it. After he did it (and given that EY doesn't respond to threats speculative:wga should act now based on the Roko incident) it isn't threat. It is then just a historical sequence of events. It wouldn't even be a particularly unique sequence events.

Wfg is far from the only person who responded by punishing SIAI in a way EY would expect to increase existential risk. ie. Not donating to SIAI when they otherwise would have.or by updating their p(EY(SIAI) is a(re) crackpot(s)) and sharing that knowledge. The description RationalWiki would be an example.

Comment author: waitingforgodel 23 September 2010 07:35:12PM 0 points [-]

These are really interesting points. Just in case you haven't seen the developments on the thread, check out the whole thing here.

I'm not sure that blackmail is a good name to use when thinking about my commitment, as it has negative connotations and usually implies a non-public, selfish nature.

I'm also pretty sure it's irrational to ignore such things when making decisions. Perhaps not in a game theory sense, but absolutely in the practical life-theory sense.

As an example, our entire legal system is based on these sorts of credible threats.

If EY feels differently I'm not sure what to say except that I think he's being foolish. I see the game theory he's pretending exempts him from considering others reactions to his actions, I just don't think it's rational to completely ignore new causal information.

But like I said earlier, I'm not saying he has to do anything, I'm just making sure we all know that an existential risk reduction of 0.0001% via LW censorship won't actually be a reduction of 0.0001%.

(and though you deleted the relevant part, I'd also be down to discuss what a sane moderation system should be like.)

Comment author: Wei_Dai 24 September 2010 08:18:21AM 14 points [-]

Suppose I were to threaten to increase existential risk by 0.0001% unless SIAI agrees to program its FAI to give me twice the post-Singuarity resource allocation (or whatever the unit of caring will be) that I would otherwise receive. Can see why it might have a policy against responding to threats? If Eliezer does not agree with you that censorship increases existential risk, he might censor some future post just to prove the credibility of his precommitment.

If you really think censorship is bad even by Eliezer's values, I suggest withdrawing your threat and just try to convince him of that using rational arguments. I rather doubt that Eliezer has some sort of unfixable bug regarding censorship that has to be patched using such extreme measures. It's probably just that he got used to exercising strong moderation powers on SL4 (which never blew up like this, at least to my knowledge), and I'd guess that he has already updated on the new evidence and will be much more careful next time.

Comment author: wedrifid 24 September 2010 08:33:28AM *  5 points [-]

If you really think censorship is bad even by Eliezer's values, I suggest withdrawing your threat and just try to convince him of that using rational arguments.

I do not expect that (non-costly signalling by someone who does not have significant status) to work any more than threats would. A better suggestion would be to forget raw threats and consider what other alternatives wfg has available by which he could deploy an equivalent amount of power that would have the desired influence. Eliezer moved the game from one of persuasion (you should not talk about this) to one about power and enforcement (public humiliation, censorship and threats). You don't take a pen to a gun fight.

Comment author: Wei_Dai 24 September 2010 10:01:57PM 2 points [-]

I don't understand why, just because Eliezer chose to move the game from one of persuasion to one about power and enforcement, you have to keep playing it that way.

If Eliezer is really so irrational that once he has exercised power on some issue, he is no longer open to any rational arguments on that topic, then what are we all doing here? Shouldn't we be trying to hinder his efforts (to "not take over the world") instead of (however indirectly) helping him?

Comment deleted 25 September 2010 12:15:18AM [-]
Comment author: Wei_Dai 25 September 2010 02:06:14AM 2 points [-]

They could talk about it elsewhere.

My understanding is that waitingforgodel doesn't particularly want to discuss that topic, but thinks that it's important that LW's moderation policy be changed in the future for other reasons. In that case it appears to me the best way to go about it is to try to convince Eliezer using rational arguments.

A public commitment has been made.

Commitment to a particular moderation policy?

Eliezer has a bias toward secrecy.

I'm inclined to agree, but do you have an argument that he is biased (instead of us)?

In my observation Eliezer becomes irrational when it comes to dealing with risk.

I'd be interested to know what observation you're referring to.

Eliezer has (much) higher status. Status drastically hinders the ability to take on board other people's ideas when they contradict your own.

True, but I've been able to change his mind on occasion (whereas I don't think I've ever succeeded in changing Robin Hanson's for example).

The most important rational arguments that weigh into the decision involve discussing the subject matter itself. This is forbidden.

I doubt that waitingforgodel has any arguments involving the forbidden topic itself. Again, he hasn't shown any interest in that topic, but just in the general moderation policy.

Overall, I agree that Eliezer is unlikely to be persuaded, but it still seems to be a better chance than anything else waitingforgodel can do.

Comment author: wedrifid 25 September 2010 08:19:37AM 0 points [-]

My understanding is that waitingforgodel doesn't particularly want to discuss that topic, but thinks that it's important that LW's moderation policy be changed in the future for other reasons.

See waitingforgodel's actual words on the subject. We could speculate that these "aren't his real reasons" but they certainly are sane reasons and it isn't usually useful to presume we know what people want despite what they say. At least for the purpose of good faith discussion if not for out personal judgement.

In that case it appears to me the best way to go about it is to try to convince Eliezer using rational arguments.

Waitingforgodel's general goals could be achieved without relying on LW itself but in a way that essentially nullifies the censorship influence (at least in an 'opt in' manner), even ensuring a negligible onging trivial inconvenience. This wouldn't be easy or likely for him to achieve but see below for a possible option. Assuming an outcome was achieved that ensured overt censorship created more discussion rather than less (Streisand Effect) it may actually become in Eliezer's interest to allow such discussions on LW. That would remove attention from the other location and put it back to a place where he can express a greater but still sub-censorship form of influence.

Commitment to a particular moderation policy?

More so on this specific topic than the general case. You are right that it wouldn't be violating a public commitment to not censor something unrelated.

Now, there is a distinction to be made that I consider important. Let's not pretend this is about moderation. Moderation in any remotely conventional sense would be something that applied to Eliezer's reply and not Roko's post. There hasn't been an instance of more dramatic personal abuse. The response was anything but 'moderate'. Without for the purpose of this point labelling it good or bad this is about censoring an idea. I don't think those who are most in support of the banning would put this in the same category as moderation.

I'm inclined to agree, but do you have an argument that he is biased (instead of us)?

Not right now but it is true that 'him or us' is something to consider if I was focusing on this issue. I actually typed some examples in the grandparent but removed them. I present the 'secrecy bias' as a premise which the reader would either share or not without getting distracted by disagreement with respect to underlying reasoning.

True, but I've been able to change his mind on occasion (whereas I don't think I've ever succeeded in changing Robin Hanson's for example).

This is true and I should take the time to say I'm impressed with how well Eliezer works to counter the status effect. This is something that is important to him and he handles it better than most.

I model Robin as an academic, actually using Robin's theories on how academics can be expected to behave to make reasonably good predictions about his behavior. It isn't often that saying someone has irrational biases actually constitutes a compliment. :)

Overall, I agree that Eliezer is unlikely to be persuaded, but it still seems to be a better chance than anything else waitingforgodel can do.

Absolutely. But then, who really thought he could significantly increase existential risk anyway?

It probably would be at least be possible for him to develop the skills and connections to create either a public place for discussion of anything and everything forbidden here and make it sufficiently visible that censorship actually increases visibility. He could even modify your new comments viewer such that it also displays discussion of the censored topics, probably highlighted for visibility.

Alternately, he could arrange a way to cooperate with other interested parties to collectively condition all their existential risk donation on certain minimum standards of behavior. This is again not coercive, just practical action. It makes a lot of sense to refrain from providing assistance to an organisation that censors all discussion regarding whether the AI they wish to create would torture people who didn't give them all their money. It also doesn't mean that the charitable contributors are committing to wasting the money on crack and hookers if they do not get their way. Contributions could be invested wisely and remain available to the first existential risk and opportunity organisation that meets a standard of predicted effectiveness and acceptable ethical behavior.

I doubt that waitingforgodel has any arguments involving the forbidden topic itself.

He probably doesn't. Others, including myself, do and analyzing the subject would produce more (for and against). This suggests that it would be a bad idea for wfg to try to persuade Eliezer with rational argument himself. If his belief is that other people have arguments that are worth hearing then he is best off seeking to find a way to make them heard.

People should never limit themselves to actions that are ineffective, except when the 'choke' effect is in play and you need to signal low status. That actually seems to be the main reason we would try to make him work within the LW power structure. We wouldn't, for example, tell Robin Hanson or even yourself that you should limit yourself to persuading the authority figures. We'd expect you to do what works.

I don't think waitingforgodel will do any of these things and we could say that he could not do these things given that the motivation to gain the skills required for that sort of social influence is a trait that few people have and wfg (and most people) is unlikely to be willing to engage in the personal development that leads him in that direction.

(Thankyou for the well reasoned and non-aggressive responses. I value being able to explore the practical implications of the issue. This strikes at the core of important elements of instrumental rationality.)

(A random note: I had to look up the name for the Streisand Effect from a comment made here yesterday by Kodos. I was surprised to discover just how many comments have been made since then. I didn't keep a count but it was a lot.)

Comment author: Wei_Dai 25 September 2010 07:46:12PM *  1 point [-]

That actually seems to be the main reason we would try to make him work within the LW power structure.

I don't think it was the main reason for my suggestion. I thought that threatening Eliezer with existential risk was obviously a suboptimal strategy for wfg, and looked for a better alternative to suggest to him. Rational argument was the first thing that came to mind since that's always been how I got what I wanted from Eliezer in the past.

You might be right that there are other even more effective approaches wfg could take to get what he wants, but to be honest I'm more interested in talking about Eliezer's possible biases than the details of those approaches. :)

Your larger point about not limiting ourselves to actions that are ineffective does seem like a good one. I'll have to think a bit about whether I'm personally biased in that regard.

Comment author: waitingforgodel 24 September 2010 09:30:30AM *  1 point [-]

Good questions, these were really fun to think about / write up :)

First off let's kill a background assumption that's been messing up this discussion: that EY/SIAI/anyone needs a known policy toward credible threats.

It seems to me that stated policies to credible threats are irrational unless a large number of the people you encounter will change their behavior based on those policies. To put it simply: policies are posturing.

If an AI credibly threatened to destroy the world unless EY became a vegetarian for the rest of the day, and he was already driving to a BBQ, is eating meat the only rational thing for him to do? (It sure would prevent future credible threats!)

If EY planned on parking in what looked like an empty space near the entrance to his local supermarket, only to discover that on closer inspection it was a handicapped-only parking space (with a tow truck only 20 feet away), is getting his car towed the only rational thing to do? (If he didn't an AI might find out his policy isn't iron clad!)

This is ridiculous. It's posturing. It's clearly not optimal.

In answer to your question: Do the thing that's actually best. The answer might be to give you 2x the resources. It depends on the situation: what SIAI/EY knows about you, about the likely effect of cooperating with you or not, and about the cost vs benefits of cooperating with you.

Maybe there's a good chance that knowing you'll get more resources makes you impatient for SIAI to make a FAI, causing you to donate more. Who knows. Depends on the situation.

(If the above doesn't work when an AI is involved, how about EY makes a policy that only applies to AIs?)

In answer to your second paragraph I could withdraw my threat, but that would lessen my posturing power for future credible threats.

(har har...)

The real reason is I'm worried about what happens while I'm trying to convince him.

I'd love to discuss what sort of moderation is correct for a community like less wrong -- it sounds amazing. Let's do it.

But no way I'm taking the risk of undoing my fix until I'm sure EY's (and LW's) bugs are gone.

Comment author: Alicorn 23 September 2010 07:41:02PM 9 points [-]

I'm not sure that blackmail is a good name to use when thinking about my commitment, as it has negative connotations and usually implies a non-public, selfish nature.

More importantly, you aren't threatening to publicize something embarrassing to Eliezer if he doesn't comply, so it's technically extortion.

Comment author: Wei_Dai 23 September 2010 07:47:59PM 5 points [-]

I think by "blackmail" Eliezer meant to include extortion since the scenario that triggered that comment was also technically extortion.

Comment author: waitingforgodel 23 September 2010 07:43:58PM -2 points [-]

That one also has negative connotation, but it's your thinking to bias as you please :p

Comment author: wedrifid 24 September 2010 04:49:39AM 2 points [-]

Technical analysis does not imply bias either way. Just curiosity. ;)

Comment author: waitingforgodel 23 September 2010 07:47:18PM 0 points [-]

To put that bit about the legal system more forcefully:

If EY really doesn't include these sorts of things in his thinking (he disregards US laws for reasons of game theory?), we have much bigger things to worry about right now than 0.0001% censorship.

Comment author: Nick_Tarleton 23 September 2010 07:47:47PM *  1 point [-]

Perhaps you weren't aware, but Eliezer has stated that it's rational to not respond to threats of blackmail.

I'm also pretty sure it's irrational to ignore such things when making decisions. Perhaps not in a game theory sense, but absolutely in the practical life-theory sense.

As an example, our entire legal system is based on these sorts of credible threats.

To be precise, not respond when whether or not one is 'blackmailed' is counterfactually dependent on whether one would respond, which isn't the case with the law. (Of course, there are unresolved problems with who 'moves first', etc.)

Comment author: waitingforgodel 23 September 2010 07:50:17PM 0 points [-]

Fair enough, so you're saying he only responds to credible threats from people who don't consider if he'll respond to credible threats?

Comment author: Nick_Tarleton 23 September 2010 07:52:53PM 2 points [-]

Yes, again modulo not knowing how to analyze questions of who moves first (e.g. others who consider this and then make themselves not consider if he'll respond).

Comment author: timtyler 24 September 2010 07:52:45PM 1 point [-]

Perhaps you weren't aware, but Eliezer has stated that it's rational to not respond to threats of blackmail.

I don't think he was talking about human beings there. Obviously you don't want a reputation for being susceptable to being successfully blackmailed, but IMHO, maximising expected utilily results in a strategy which is not as simple as never responding to blackmail threats.

Comment author: khafra 24 September 2010 08:29:46PM 1 point [-]

I think this is correct. Eliezer's spoken from The Strategy of Conflict before, which goes into mathematical detail about the tradeoffs of precommitments against inconsistently rational players. The "no blackmail" thing was in regards to a rational UFAI.

Comment author: khafra 23 September 2010 01:05:30PM 2 points [-]

Ironically, your comment series is evidence that censorship partially succeeded in this case. Although existential risk could increase, that was not the primary reason for suppressing the idea in the post.

Comment author: timtyler 23 September 2010 08:14:17PM 3 points [-]

Succeeded - in promoting what end?

Comment author: kodos96 24 September 2010 03:44:54AM 1 point [-]
Comment author: wedrifid 24 September 2010 05:00:52AM 8 points [-]

I've actually speculated as to whether Eliezer was going MoR:Quirrel on us. Given that aggressive censorship was obviously going to backfire a shrewd agent would not use such an approach if they wanted to actually achieve the superficially apparent goal. Whenever I see an intelligent, rational player do something that seems to be contrary to their interests I take a second look to see if I am understanding what their real motivations are. This is an absolutely vital skill when dealing with people in a corporate environment.

Could it be the case that Eliezer is passionate about wanting people to consider torture:AIs and so did whatever he could to make it seem important to people, even though it meant taking a PR hit in the process? I actually thought this question through for several minutes before feeling it was safe to dismiss the possibility.

Comment author: kodos96 24 September 2010 05:58:18AM 1 point [-]

So I actually haven't read MoR - could you summarize the reference for me? I mean, I can basically see what you're saying from context, but is there anything beyond that it would be useful to know?

My instinct is that it just doesn't feel like something Eliezer would do. But what do I know?

Comment author: wedrifid 24 September 2010 06:11:52AM 3 points [-]

So I actually haven't read MoR - could you summarize the reference for me? I mean, I can basically see what you're saying from context, but is there anything beyond that it would be useful to know?

There isn't much more to it than can be inferred from the context. MoR:Quirrel is just a clever, devious and rational manipulator.

My instinct is that it just doesn't feel like something Eliezer would do. But what do I know?

I don't either... but then that's the assumption MoR:Harry made about MoR:Dumbledore. At Quirrel's prompting Harry decided "it was time and past time to ask Draco Malfoy what the other side of that war had to say about the character of Albus Percival Wulfric Brian Dumbledore." :)

(Of course EY hasn't been in a war and I don't think there are any people who accuse him of being an especially devious political manipulator.)

Comment author: lessdazed 16 August 2011 06:23:56AM 0 points [-]

I had thought about it and reached no conclusion.