LESSWRONG
LW

Comment Permalink

I had no intention of doing anyone any harm.

I know.

could you please stop saying that I advocate killing people?

reviews my wording very carefully

"If virtualizing people is violence ... Wei_Dai ... seems to be advocating "

"Advocating for an AGI that will kill all of humanity (in the context of this is not what you said) vs. advocating for an AGI that could kill all of humanity (context: this is what you said)"

My understanding is that it's your perspective that copying people and removing the physical original might not be killing them, so my statements reflect that but maybe it would make you feel better if I did this:

"If virtualizing people is violence ... Wei_Dai ... seems to be advocating ... kill the entire population of earth (though he isn't convinced that they would die)"

And likewise with the other statement.

Sorry for the upset that has probably caused. It wasn't my intent to accuse you of actually wanting to kill everyone. I just disagree with you and am very concerned about how your statement looks to others with my perspective. More importantly, I feel concerned about the existential risk if people such as yourself (who are prominent here and connected with SIAI) are willing to have an AGI that could (in my view) potentially kill the entire human race. My feeling is not that you are violent or intend any harm, but that you appear to be confused in a way that I deem dangerous. Someone I'm close to holds a view similar to yours and although I find this disturbing, I accept him anyway. My disagreement with you is not personal, it's not a judgment about your moral character, it's an intellectual disagreement with your viewpoint.

As I clarified in a subsequent comment in that thread, "if the FAI concludes that replacing a physical person with a software copy isn't a harmless operation, it could instead keep physical humans around and place them into virtual environments Matrix-style."

I think the purpose of this part is to support your statement that you have no intention to harm anyone, but if it's an argument against some specific part of my comment, would you mind matching them up because I don't see how this refutes any of my points.

I've never received any money from them and am not even a Research Associate. I have independently done work that may be useful for SIAI, but I don't think that's the same thing from a PR perspective.

It's not easy for me to determine your level of involvement from the website. This here suggests that you've done important work for SIAI:

Vladimir Nesov, a decision theory researcher, holds an M.S. in applied mathematics and physics from Moscow Institute of Physics and Technology. He has worked on Wei Dai’s updateless decision theory, in pursuit of one of the Singularity Institute’s core research goals: that of developing a “reflective decision theory.”

http://singularity.org/blog/2011/07/22/announcing-the-research-associates-program/

If one is informed of the exact relationship between you and SIAI, it is not as bad, but:

A. If someone very prominent on LessWrong (a top contributor) who has been contributing to SIAI's decision theory ideas (independently) does something that looks bad, it still makes them look bad.

B. The PR effect for SIAI could be much worse considering that there are probably lots of people who read the site and see a connection there but do not know the specifics of the relationship.

"Let's work out all the problems involved in letting the AGI decide what is ethical."

Okay but how will you know it's making the right decision if you do not even know what the right decision is for yourself? If you do not think it is safe to simply give the AGI an algorithm that looks good without testing to see whether running the algorithm outputs choices that we want it to make, then how do you test it? How do you even reason about the algorithm? How do you make those beliefs "pay rent", as the sequence post puts it?

I see now that the statement could be interpreted in one of two ways:

"Let's work out all the problems involved in letting the AGI define ethics."

"Let's work out all the problems involved in letting the AGI make decisions on it's own without doing any of the things that are wrong by our definition of what's ethical."

Do you not think it better to determine for ourselves whether virtualizing everyone means killing them, and then ensure that the AGI makes the correct decision? Perhaps the reason you approach it this way is because you don't think it's possible for humans to determine whether virtualizing everyone is ethical?

I do think it is possible, so if you don't think it is possible, let's debate that.

Wei Dai12y10

Perhaps the reason you approach it this way is because you don't think it's possible for humans to determine whether virtualizing everyone is ethical?

I think it may not be possible for humans to determine this, in the time available before someone builds a UFAI or some other existential risk occurs. Still, I have been trying to determine this, for example just recently in Beware Selective Nihilism. Did you see that post?

Were you serious about having Eliezer censor my comment? If so, now that you have a better understanding of my ideas and relationship w... (read more)

See in context

43 New censorship: against hypothetical violence against identifiable people

by Eliezer Yudkowsky

23rd Dec 2012

1 min read

448

43

New proposed censorship policy:

Any post or comment which advocates or 'asks about' violence against sufficiently identifiable real people or groups (as opposed to aliens or hypothetical people on trolley tracks) may be deleted, along with replies that also contain the info necessary to visualize violence against real people.

Reason: Talking about such violence makes that violence more probable, and makes LW look bad; and numerous message boards across the Earth censor discussion of various subtypes of proposed criminal activity without anything bad happening to them.

More generally: Posts or comments advocating or 'asking about' violation of laws that are actually enforced against middle-class people (e.g., kidnapping, not anti-marijuana laws) may at the admins' option be censored on the grounds that it makes LW look bad and that anyone talking about a proposed crime on the Internet fails forever as a criminal (i.e., even if a proposed conspiratorial crime were in fact good, there would still be net negative expected utility from talking about it on the Internet; if it's a bad idea, promoting it conceptually by discussing it is also a bad idea; therefore and in full generality this is a low-value form of discussion).

This is not a poll, but I am asking in advance if anyone has non-obvious consequences they want to point out or policy considerations they would like to raise. In other words, the form of this discussion is not 'Do you like this?' - you probably have a different cost function from people who are held responsible for how LW looks as a whole - but rather, 'Are there any predictable consequences we didn't think of that you would like to point out, and possibly bet on with us if there's a good way to settle the bet?'

Yes, a post of this type was just recently made. I will not link to it, since this censorship policy implies that it will shortly be deleted, and reproducing the info necessary to say who was hypothetically targeted and why would be against the policy.

Censorship

Personal Blog

43

New censorship: against hypothetical violence against identifiable people

-1All_work_and_no_play

New Comment

448 comments, sorted by

top scoring

Click to highlight new comments since: Today at 4:11 PM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-][anonymous]12y570

I'm started to feel strongly uncomfortable about this, but I'm unsure if that's reasonable. Here's some arguments ITT that are concerning me:

Does advocating gun control, or increased taxes, count? They would count as violence is private actors did them, and talking about them makes them more likely (by states).

Violence is a very slippery concept. Perhaps it is not the best one to base mod rules on. (more at end)

We're losing Graham cred by being unwilling to discuss things that make us look bad.

This one is really disturbing to me. I don't like all the self-conscious talk about how we are percieved outside. Maybe we need to fork LW, to accomplish it, but I want to be able to discuss what's true and good without worrying about getting moderated. My post-rationality opinions have already diverged so far from the mainstream that I feel I can't talk about my interests in polite society. I don't want this here too.

If I see any mod action that could be destroyed by the truth, I will have to conclude that LW management is borked and needs to be forked. Until then I will put my trust in the authorities here.

Would my pro-piracy arguments be covered by this? What about my pro-coup d

... (read more)

[-]AlexMennen12y250

I support censorship, but only if it is based on the unaccountable personal opinion of a human. Anything else is too prone to lost purposes. If a serious rationalist (e.g. EY) seriously thinks about it and decides that some post has negative utility, I support its deletion. If some unintelligent rule like "no hypothetical violence" decides that a post is no good, why should I agree? Simple rules do not capture all the subtlety of our values; they cannot be treated as Friendly.

It makes sense to have mod discretion, but it also makes sense to have a list of rules that the mods can point to so that people whose posts get censored are less likely to feel that they are being personally targeted.

[-][anonymous]12y160

Yes. Explanatory rules are good. Letting the rules drive is not.

[-]Eliezer Yudkowsky12y240

These are explanations, not rules, check.

3Luke_A_Somers12y

Hence "may at the admins' option be censored"

[-]handoflixue12y120

If I see any mod action that could be destroyed by the truth, I will have to conclude that LW management is borked and needs to be forked. Until then I will put my trust in the authorities here.

I want to upvote these two sentences again and again.

5Multiheaded12y

I think that there's the usual paradox of benevolent dictatorship here; you can only trust humans who clearly don't seek this position for selfish ends and aren't likely to present a rational/benevolent front just so you would give them political power. In a liberal/democratic political atmosphere, self-proclaimed benevolent dictators are a rare and prized resource; you can pressure one to run a website, an organization, etc to the best of their ability. But if dictatorship were to be seen as the norm, and you couldn't easily fall back on democracy, rule by committee, anarchy, etc, and had to choose between a few dictators, then the standards of dictatorial control would surely plummet and it would be psychologically much more difficult to change the form of organization. So, IMO, isolated experiments with dictatorship are fine; overall preference for it is terribly dangerous. (All of the above goes only for humans, of course; I have no qualms about FAI rule.) P.S.: I googled for "benevolent dictator" + "paradox" and found an argument similar to mine.

1[anonymous]12y

Interesting. Do you think there are dictator-selection procedures that don't have either set of failure modes (selecting for looks/promises to loot the commons/lack of leadership, selecting for power-hungry tyrants)?

4Multiheaded12y

Only a single one: a great actually-benevolent-dictator, with a good insight into people and lots of rationality, personally selects his successor among several candidates, after lengthy consideration and hidden testing. But, of course, remove one of the above qualifiers, and it can blow up regardless of the first dictator's best intentions. See e.g. Marcus Aurelius and Commodus. So, on a meta level, no, there's likely no system that would work for humans. (I think that "real" democracy is also too dangerous - see the 19th and early 20th century - so either some form of sophisticated rule by committee or a state of anarchy could be the safest option for baseline humanity.)

3[anonymous]12y

What about technocracy a-la china? And FAI, obviously. Really? Safe in the sense of "too incompetent to execute a mass-murder"? Also, anarchy is a military vacuum.

4Eliezer Yudkowsky12y

Then discussing it on the public Internet is the wrong thing to do. I can't compare it to anything but juvenile male locker-room boasting.

[-][anonymous]12y260

What if you aren't sure if violence is the right thing to do? You obviously should want as many eyeballs to debug your thinking on that as possible no?

4Plasmon12y

If you actually believe that violence might be the right thing to do, then you assign non-negligible probability to * the discussion will convince you that violence is indeed the right thing to do * you now have moral imperative to do violence, and you will act on this or convince others to act on it * you will want the discussion to never have occurred in the first place, because authorities can use it to track you down , and suppress your justified violence If you want to discuss a coup or something do it in a less easily traceable fashion (not on a public forum. Use encryption. ).

[-][anonymous]12y130

You do realize this argument generalizes to discussing many things beyond violence right? So if this is your true rejection I hope you've spent some time decompartmentalizing on this.

7[anonymous]12y

I don't see how to decompartmentalize that, so I'm interested in what you are referring to.

[-][anonymous]12y100

The thing is discussing desirability of violence and carrying out violence are not necessarily done by the same person. Indeed historically they usually aren't. This does not remove moral culpability but does provide some legal protection.

-1Plasmon12y

Certainly. I consider this to be evidence that the people discussing the desirability of violence do not actually believe what they are saying. They are merely attempting to raise their status in an in-group which hates the group against which violence is being discussed. Due to hate speech laws, you may have less legal protection than you expect.

9[anonymous]12y

This fits with gwern's model of terrorist groups as not being about political objectives but about dysfunctional support groups of people who bully each other into action because of all too human social games. But you are making a mistake here. A similar one that people make after hearing about the evolutionary origins of altruism and then go on to behave as if altruism doesn't really exist. Like thinking a mother was really doing fitness maximizing calculations when deciding to give the runt cub less food than the strong ones. She just feels less inclined to give it food because it isn't as cute or something. The mechanism that produced that feeling certainly was optimized with fitness maximization as a goal in the past but that isn't what is going on in her brain. I'm pretty sure Ayatollah Khomeini, Thomas Paine or Lenin probably honestly believed in the desirability of the violence they where promoting. They weren't faking it. But I think they probably did believed in it because of the social reasons you mention.

2Plasmon12y

The fitness maximising calculations are encoded, by evolution, in the neural patterns relating to the cuteness response. The individuals whose cuteness response correlates with fitness are themselves more fit. Those who would give more food to their malformed three-legged offspring will go extinct. So of course the mother is doing fitness calculations. It doesn't feel like that from the inside, but that doesn't make it any less true. And the religions people who merely believe that they believe in their god, they feel very religions from the inside. There is a distinction to be made between the internal state of they who argue violence against a certain group is a good thing, but don't lift a finger themselves (BELIEF1) , and, they who upon being convinced that violence against this group is a good thing, actually attack said group (BELIEF2). It is that distinction that I mean when I say they don't really believe, even though both feel like honest belief from the inside. Neither is faking it, but there's still a distinction.

4[anonymous]12y

No she is not. Except in the sense that she is perhaps one small step in something that does fitness calculations, but looking at her brain you wouldn't find fitness maximization calculations being done just execution of old adaptations. Religious people believe they believe in God. And many of them are correct on this. It can also be called division of labour. My comparative advantage may lie in bashing Wiggin heads or crafting arguments for why bashing Wiggin heads is good or organizing the logistics so our heads don't get bashed by Wiggins so that we can bash more of theirs. I don't see from a consquentalist stand point what is so different between me pyhsically bashing a Wiggin head, pressing a button that activates a machine that bashes as Wiggin head and manipulating someone into bashing a Wiggin head. To call only one of these indication of "real" belief that bashing Wiggin heads is good (and I hope we all agree it is!) seems not very useful at all, especially since it is perfectly possible that the preferences are literally identical inside their brains but merely the means available to them are what varies. The distinction seems useful only in very peculiar circumstances, like trying to discover my preferences with regards to personal physical confrontation or combat.

0Plasmon12y

These old adaptations encode rough heuristics of limited applicability which approximate fitness calculations (if the environment has been fairly constant for long enough). They are actually there inside the brain. What else do you think the cuteness response as you used here is? Are they? So very few of them actually take their beliefs seriously. So very few of them actually behave as if their expected utility calculations are dominated by treats of eternal damnation and promises of eternal salvation.

0Plasmon12y

Yes. The problem is that this is exactly the rationalisation that someone would use if it weren't true. Then again, it might be true. We need to distinguish * (type A) Someone wants to rise in power within a certain group, advocates violence against a hated out-group, and remains largely protected from legal consequences himself because he doesn't actually commit any violent acts. When asked, he claims his non-action is due to division-of-labour-reasoning. * (type B) Someone actually thinks violence against a certain out-group is a good thing (in the greater good sense), and doesn't commit any violent acts himself based on division-of-labour-reasoning. When asked about his motivations, he is not (easily?) distinguishable from (A). What's the difference? The difference is that (type A) should be discouraged from encouraging violence. If a (type A) successfully encourages a group of followers to commit violence against a hated out-group , people get hurt. This was not the (type A)'s intention, it's just an unfortunate side effect that he doesn't really care about. (type B)s, on the other hand, should be listened to, and their arguments weighed carefully. For the greater good, you know. In fact this seems like a good reason for (type B)s to signal that they themselves do not in any way profit from the violence. What are your priors? More (type B)? More (type A)? You said it yourself : not being the one who actually commits the violent acts provides some legal protection. Your not ending up in jail is a consequence. (I don't actually know what a Wiggin head is, I assume "bashing a Wiggin head" is some socially unaccepted form of violence).

1Viliam_Bur12y

See here: wiggin

6[anonymous]12y

lol. Hate speech laws are primarily about punishing ethnocentric white people, secondarily for protecting very specific minorities. Even when written so as to protect people of a certain profession or class or education level or political ideology as they are in my country they are never used that way. An example: Do you think that saying you want to take stuff from or harm rich people without getting into specifics about a particular person will ever get you into legal trouble?

0TheOtherDave12y

I imagine it depends a lot on the extent to which the legal jurisdiction I'm in at the time is influenced by rich people, and the extent to which those rich people take my having said that seriously. In most jurisdictions and for most audiences, very likely not, unless I'm a far more compelling speaker than I think I am.

0Plasmon12y

I was talking in general, not about you specifically. In fact I much appreciate your out-of-the-box view on many subjects, and I can guess why you would argue against any form of censorship here, slippery slopes and all that. Example of hate-speech laws being used

5[anonymous]12y

I think your example is rather atypical to be honest at least in the wider West. Emma West being the more typical one. Very much like with hate crime laws there is controversy whether hate speech against white people even is hate speech. What would be considered unacceptable for one group is not unacceptable for another. The star of the recent popular movie Django Unchained, Jamie Fox joked for example: in light of his other comments this is interesting Whether this combined is ominous, righteous or innocuous depends on your model of the world. That how such laws are applied depends heavily on what kind of model of the world judges or police officers are likely to use is hardly disputable however.

4Plasmon12y

Oh, I agree fully that such laws are problematic and open to abuse, and that it might well be better for no such laws to exist at all. Nonetheless they exist and should occur as a (possibly very low) cost in the calculation of the expected utility of advocating violence.

0Eugine_Nier12y

Not necessarily. It could be division of labor since the people who are good at figuring out which violence to do are not necessarily the same people good at doing violence.

[-]DataPacRat12y160

A friend and I once put together a short comic trying to analyze democracy from an unusual perspective, including presenting the idea that an underlying threat of violent popular uprising should the system be corrupted helps keep it running well. This was closely related to a shorter comic presenting some ideas on rationality. The project led to some interesting discussions with interesting people, which helped me figure out some ideas I hadn't previously considered, and I consider it to have been worth the effort; but I'm unsure whether or not it would fall afoul of the new policy.

How 'identifiable' do the targets of proposed violence have to be for the proposed policy to apply, and how 'hypothetical' would they have to be for it not to? Some clarification there would be appreciated.

0MugaSofer12y

It's only applied if a mod feels like it.

3Kawoomba12y

Also, implying that violence is best discussed in private, versus not being discussed at all. It's like saying in public "But let's talk about our illegal activities in a more private venue." There should be no perception of LW being associated with such, period (.)

1AdeleneDawner12y

Actually, I can think of at least one type of situation where this isn't true, though it seems unwise to explain it in public and in any case it's still not something you'd want associated with LW, or in fact happening at all in most cases.

0MugaSofer12y

What if someone is, y'know, unsure?

7Eliezer Yudkowsky12y

Generally speaking, there's a lot of options grownups in real life resort to before they resort to violence, and I would have no problem with a post describing the fully generic considerations and how far you'd actually have to go down the decision tree before you got to violence, without any identifiables being named. People who honestly don't realize this would be welcome to read that post. I may be somewhat prejudiced by considering it completely obvious that jumping straight to violence as a cognitive answer and then blathering about your conspiracy on the Internet is merely stupid.

2MugaSofer12y

That ... doesn't seem to answer my question. Perhaps an example is in order. Someone lives in an area where there recently been a number of violent muggings. They are considering bringing a gun with them when they go out, in order to defend themself; they suspect they may be overestimating the danger based on news reports. So they decide to ask here if there are any relevant biases that may be coloring their judgement., and this leads into a general discussion of what chance there should be of encountering violent criminals before it becomes rational to arm yourself (and risk accidentally injuring or killing yourself or passersby.) Does this help clarify my problem?

7wedrifid12y

Discussion on lesswrong is not likely to give them an answer. Honestly, I can't think of any public place on the internet that is likely to be all that helpful, unfortunately.

-1[anonymous]12y

Good point.

[-]quintopia12y450

EY has publicly posted material that is intended to provoke thought on the possibility of legalizing rape (which is considered a form of violence). If he believed that there was positive utility in considering such questions before, then he must consider them to have some positive utility now, and determining whether the negative utility outweighs that is always a difficult question. This is why I will be opposed to any sort of zero tolerance policy in which the things to be censored is not well-defined a definite impediment to balanced and rationally-considered discussion. It's clear to me that speaking about violence against a particular person or persons is far more likely to have negative consequences on balance, but discussion of the commission of crimes in general seems like something that should be weighed on a case-by-case basis.

In general, I prefer my moderators to have a fuzzy set of broad guidelines about what should be censored in which not deleting is the default position, and they actually have to decide that it is definitely bad before they take the delete action. The guidelines can be used to raise posts to the level of this consideration and influence their judgment on this decision, but they should never be able to say "the rules say this type of thing should be deleted!"

9Error12y

I'm not sure how this is relevant; there's a good bit of difference between discussion of breaking a law and discussion of changing it. That said, I think I'm reading this differently than most in the thread. I'm understanding it as aimed against hypotheticals that are really "hypotheticals". In answer to the question that was actually asked in the post, here is a non-obvious consequence: My impression of the atheist/libertarian/geek personspace cluster that makes up much of LW's readership is that they're generally hostile to anything that smells like conflating "legal" with "okay"; and also to the idea that they should change their behavior to suit the rest of the world. You might find you're making LW less off-putting to the mainstream at the cost of making it less attractive to its core audience. (but you might consider it worth that cost) As both a relatively new contributor and a member of said cluster, this policy makes me somewhat uncomfortable at first glance. Whether that generalizes to other potential new contributors, I cannot say. I present it as proof-of-concept only.

3A1987dM12y

IAWYC, but that was a story set in the far future with a discussion that makes clear (to me at least) that our present is so different from that that the author wouldn't ever even dream of suggesting to do anything remotely like that in our times. It isn't remotely similar to (what Poe's Law predicts people will get from) the recent suggestion about tobacco CEOs.

3Eliezer Yudkowsky12y

That's an... interesting way of putting it, where by "interesting" I mean "wrong". I could go off on how the idea is that there's particular modern-day people who actually exist and that you're threatening to harm, and how a future society where different things feel harmful is not that, but you know, screw it. The 'rules' do not 'mandate' that I delete anything. They hardly could. I'm just, before I start deleting things, giving people fair notice that this is what I'm considering doing, and offering them a chance to say anything I might have missed about why it's a terrible idea.

[-]wedrifid12y620

That's an... interesting way of putting it, where by "interesting" I mean "wrong".

If you genuinely can't see how similar considerations apply to you personally publishing rape-world stories and the reasoning you explicitly gave in the post then I suggest you have a real weakness in evaluating the consequences of your own actions on perception.

I could go off on how the idea is that there's particular modern-day people who actually exist and that you're threatening to harm, and how a future society where different things feel harmful is not that, but you know, screw it.

I approve of your Three Worlds Collide story (in fact, I love it). I also approve of your censorship proposal/plan. I also believe there is no need to self censor that story (particularly at the position you were when you published it). That said:

This kind of display of evident obliviousness and arrogant dismissal rather than engagement or---preferably---even just outright ignoring it may well do more to make Lesswrong look bad than half a dozen half baked speculative posts by CronoDAS. There are times to say "but you know, screw it" and "where by interesting I mean wrong" but those times don't include when concern is raised about your legalised-rape-and-it's-great story in the context of your own "censor hypothetical violence 'cause it sounds bad" post.

-3MugaSofer12y

So if I suggest killing people in the context of futurism, that's OK with you?

3Eliezer Yudkowsky12y

This seems to me like a deliberate misunderstanding. But taking it at face value, a story in which violence is committed against targets not analogous to any present-day identifiable people, or which is not committed for any reasons obviously analogous to present-day motives, is fine. The Sword of Good is not advocating for killing wizards who kill orcs, although Dolf does get his head cut off. Betrayed-spouse murder mysteries are not advocating killing adulterers - though it would be different if you named the victim after a specific celebrity and depicted the killer in a sympathetic light. As much as people who don't like this policy, might wish that it were impossible for anyone to tell the difference so that they could thereby argue against the policy, it's not actually very hard to tell the difference.

2kodos9612y

I didn't interpret CronoDAS's post as intending to actually advocate violence. I viewed it as really silly and kind of dickish, and a good thing that he ultimately removed it, but an actual call to violence? No. It was a thought experiment. His thought experiment was set in the present day, while yours was set in the far future, but other than that I don't see a bright line separating them. It may not be be very hard for you to tell the difference, since you wrote the policy, so you may very well have a clear bright line separating the two in your head, but we don't.

-1MugaSofer12y

I was unsure if people who do not currently exist might also be considered "identifiable real-world individuals", if discussed in the context of futurism. Thank you for clarifying.

0prase12y

He was in a different position then. Trying to gain reputation for being an original thinker requires different public outputs than attempting to earn mainstream recognition of the origanisation one is the head of.

-3jimrandomh12y

This looks like a complete misinterpretation, albeit one I've seen several times. The context of this is the novella Three Worlds Collide. (Spoilers follow). In that story humans meet two races of aliens with incompatible values, the babyeaters and the superhappies. The superhappies demand to modify human values to be more compatible with their own; and the author's perspective is that this would be a very bad thing, worth sacrificing billions of lives to prevent. This is the focus of the story. Then we find out that in this universe, rape has been legalized, and it's only a little more than a throwaway remark. What are we to make of this? Well, it's a concrete example of why changing human values would be bad. Which, given the overall story, seems like the obvious intended interpretation. But hey, male author mentioning rape - let's all be offended! His condemnation of it wasn't strong enough!

0Eugine_Nier12y

Who said anything about being offended?

[-][anonymous]12y400

Fun Exercise

Posts or comments advocating or 'asking about' violation of laws that are actually enforced against middle-class people (e.g., kidnapping, not anti-marijuana laws) may at the admins' option be censored on the grounds that it makes LW look bad and that anyone talking about a proposed crime on the Internet fails forever as a criminal

Consider what would have been covered by this 250, 100 and 50 years ago.

Bonus Consider what wouldn't have been covered by this 250, 100 and 50 years ago but would be today.

[-]Qiaochu_Yuan12y190

I see the point you're trying to make, but I don't think it constitutes a counterargument to the proposed policy. If you were an abolitionist back when slavery was commonly accepted, it would've been a dumb idea to, say, yell out your plans to free slaves in the Towne Square. If you were part of an organization that thought about interesting ideas, including the possibility that you should get together and free some slaves sometime, that organization would be justified in telling its members not to do something as dumb as yelling out plans to free slaves in the Towne Square. And if Ye Olde Eliezere Yudkowskie saw you yelling out your plans to free slaves in the Towne Square, he would be justified in clamping his hand over your mouth.

[-][anonymous]12y180

It wouldn't be dumb to argue for the moral acceptability of freeing slaves (even by force) however.

[-]Qiaochu_Yuan12y110

It wouldn't be dumb for an organization to decide that society at large might be willing to listen to them argue for the moral acceptability of freeing slaves, even by force. It would be dumb for an organization to allow its individual members to make this decision independently because that substantially increases the probability that someone gets the timing wrong.

[-]prase12y160

Beware selective application of your standards. If the members can't be trusted with one type of independent decision, why they can be trusted with other sorts of decisions?

5Qiaochu_Yuan12y

Because the decision to initiate a particular kind of public discussion entails everyone else in the organization taking on a certain level of risk, and an organization should be able to determine what kinds of communal risk it's willing to allow its individual members to force on everyone else. There are jurisdictions where criminal incitement is itself a crime.

0prase12y

I can't say whether I agree or disagree until you precise the meaning of the qualifiers "particular" and "certain". But my question was in any case probably directed a bit elsewhere: if the members shouldn't be free to write about certain class of topics because they may misjudge how the society at large would react, doesn't it imply that they shouldn't be free to write about anything because they may misjudge what the society at large might think? If the rationale is that which you say, returning back from abolitionists to LW, shouldn't the policy be "any post that is in conflict with LW interest can be deleted" rather than the overly specific rule concerning violence and only violence?

5Qiaochu_Yuan12y

"Criminal incitement" and "the risk of being arrested," then. In other time periods, substitute "blasphemy" and "the risk of being burned at the stake." They shouldn't be free to write about certain topics with the name of their organization attached to that writing, which is the case here. They can write about anything they want anonymously and with no organization's name attached because that doesn't entail the other members of the organization taking on any risk. Sure.

0Decius12y

How does an organization make decisions independently of the members of the organization?

1Qiaochu_Yuan12y

It doesn't. The distinction is between decisions that individual members make independently and decisions that individual members make communally. If it helps, the underlying moral principle I'm working from here is "try to avoid making decisions that entail other people taking on risks without their consent."

0Decius12y

Did they take on the risks when they entered the conspiracy, or do they only take on those risks when events beyond their control happen? It would be foolish to conspire with foolish or rash people, which is one reason why I don't.

-4Multiheaded12y

But I thought you'd support American slavery on general reactionary grounds? That the slaves were just wonderfully happy and content until religiously-minded Abolitionist meddlers tried to teach them to read, to disrespect their masters, etc? (semi-trolling)

[-]ChristianKl12y100

Bonus:

Consider what's likely to be covered 50 years in the future.

6Eugine_Nier12y

For something like that, consider the algorithm you use to answer it. Then consider why the output of said algorithm should at all correlate with future social trends.

2[anonymous]12y

I considered adding that too. :)

[-][anonymous]12y390

Would my pro-piracy arguments be covered by this? What about my pro-coup d'état ones?

[-][anonymous]12y240

Possibly. I hope not. I'm all for mod action, but not at the expense of political diversity.

8Jabberslythe12y

I think piracy cases are pretty similar to marijuana cases (they are even less likely to be enforced actually) which he said won't be banned.

[-]Eugine_Nier12y100

I don't think Konkvistador was talking about software piracy.

2Jabberslythe12y

Hahaha, whoops.

4A1987dM12y

You mean copyright piracy or sea piracy?

[-][anonymous]12y330

Sea piracy obviously. What kind of a person do you think I am?!

[-]Qiaochu_Yuan12y100

As someone unfamiliar with your views, I can't tell whether this is sarcasm or not, especially because of the interrobang. Can you clarify? Is there anywhere on the internet where your views are concisely summarized? (Is it in any way associated with your real name?)

[-][anonymous]12y150

The levels can be hard to disambiguate so I sympathize. I'll write my opinions out unironically. You can find the full arguments in my comment history (I can dig links to that up too).

I'm assuming you are familiar with the arguments for efficent charity and optimal employment? If not I can provide citations & links. I don't think Sea Piracy as a means to funding efficient charity is obviously worse from a utilitarian perspective than a combo with many legal professions. It may or may not be justified, I'm leaning towards it being justified on the same utilitarian grounds as government taxation can be. If not cheating on taxes to fund efficient charity is a pretty good idea. Some people's comparative advantage will lay in sea piracy.
Violating copyright on software or media products in the modern West is in general not a bad thing. But indiscriminately pirating everything may be bad.

In the grandfather comment I was aiming for ambiguity and humour.

[-]MBlume12y170

I mean, assuming that sea piracy to fund efficient charity is good, media piracy to save money that you can give to efficient charity is just obviously good.

[-][anonymous]12y130

media piracy to save money that you can give to efficient charity

Is so incredibly obviously good that I'm mystified no one is promoting it. I think the main reason is because it is "illegal".

3FiftyTwo12y

We often seperate endorsing things from believing they are good, as endorsing them implies you would like them to be prevalent which leads to collective action issues. (E.g. I think it is ok to occasionally take more than your share share of the cake if you're hungry, I wouldn't encourage it as then there wouldn't be any cake left)

2A1987dM12y

Because few people actually spend much money on copyrighted stuff they could pirate instead these days, so it's just assumed that anyone trying to do efficient charity already has more money than if they had paid for all the copyright media they've consumed?

4[anonymous]12y

Efficient charity folks are really serious about morality though and generally well off. They may have compartmentalized deontological beliefs to buy media material you consume. I bet they are more likely to pay for copyrighted works than the average person. And if you are going out to the movies or buying popular books off Amazon you should be reminded to pirate more.

[-]ChristianKl12y310

I am asking in advance if anyone has non-obvious consequences they want to point out or policy considerations they would like to raise.

I'm not sure what's obvious for you. In an enviroment without censorship you don't endorse a post by not censoring the post. If you however start censoring you do endorse a post by letting it stand.

Your legal and PR obligations for those posts that LessWrong hosts get bigger if you make editorial censorship decisions.

7David_Gerard12y

AIUI this is legally true: CDA section 230, mere hosting versus moderation.

2Viliam_Bur12y

Is there any way out of this dilemma? For example having a policy where moderator flips a coin for each offending article or comment, and head = delete, tails = keep. :D

0jefftk12y

While I don't know about the legality, practically what this does is add noise to the moderation signal. Posts that remain are still more likely to be ones that the moderator approves of, but might not be. This is actually very similar to the current system, with the randomness of coin flipping substituted for the semi-randomness of what the moderator happens to see.

[-]CronoDAS12y310

My post was indeed inappropriate. I have used the "Delete" function on it.

[-]Suryc1112y290

I'm disappointed by EY's response so far in this thread, particularly here. The content of the post above in itself did not significantly dismay me, but upon reading what appeared to be a serious lack of any rigorous updating on the part of EY to--what I and many LWers seemed to have thought were--valid concerns, my motivation to donate to the SI has substantially decreased.

I had originally planned to donate around $100 (starving college student) to the SI by the start of the new year, but this is now in question. (This is not an attempt at some sort of blackmail, just a frank response by someone who reads LW precisely to sift through material largely unencumbered by mainstream non-epistemic factors.) This is not to say that I will not donate at all, just that the warm fuzzies I would have received on donating are now compromised, and that I will have to purchase warm fuzzies elsewhere--instead of utilons and fuzzies all at once through the SI.

[-]drethelin12y190

This is similar to how I feel. I was perfectly happy with his response to the incident but became progressively less happy with his responses to the responses.

7orthonormal12y

There is a rare personality trait which allows a person to read and respond to hundreds of critical comments without compromising their perspicacity and composure. Luke, for instance, has demonstrated this trait; Eliezer hasn't (to the detriment of this discussion and some prior ones). (I'd bet at 10-to-1 that Eliezer agrees with this assessment.)

5lukeprog12y

Not sure I totally agree. My LW comments may show retained composure in most cases, but I can think of two instances in the past few months in which I became (mildly) emotional in SI meetings in ways that disrupted my judgment until after I had cooled down. Anna can confirm, as she happens to have been present for both meetings. (Eliezer could also confirm if he had better episodic memory.) The first instance was a board meeting at which we discussed different methods of tracking project expenses, the second was at a strategy meeting which Anna compared to a Markov chain. Anyway, I'm aware of people who are better at this than I am, and building this skill is one of my primary self-improvement goals at this time.

4orthonormal12y

I appreciate you sharing this. Keeping one's composure in person and keeping one's composure on the Internet are distinct aptitudes (and only somewhat correlated, as far as I can tell), and it still looks to me like you've done well at the latter.

-1wedrifid12y

There is another personality trait (or skill) that allows one to be comfortable with acknowledging areas of weakness and delegating to people more capable. Fortunately Eliezer was able to do this with respect to managing SIAI. He seems to have lost some of that capability or awareness in recent times.

0hairyfigment12y

Quite possibly. I'd still oppose putting you in charge of the website. :)

-2wedrifid12y

Irrelevant and unnecessary. Particularly since I happen to have the self awareness in question, know that becoming Luke would be exhausting to me and so would immediately hand off the responsibility. What I wouldn't do is go about acting how I naturally wish to act and be unable to comprehend why my actions have the consequences that they inevitably have. Why are you smiling? This makes your remark all the more objectionable (in as much as it indicates either wit or rapport, neither of which are present.)

0[anonymous]12y

Punishment received! Brain has learned to stop responding to responses.

[-]NancyLebovitz12y240

Posts or comments advocating or 'asking about' violation of laws that are actually enforced against middle-class people (e.g., kidnapping, not anti-marijuana laws) may at the admins' option be censored on the grounds that it makes LW look bad

I'm dubious about this because laws can change. I'm also sure I don't have a solid grasp of which laws can be enforced against middle-class people, but I do know that they aren't all like laws against kidnapping. For example, doctors can get into trouble for prescribing "too much" pain medication.

5A1987dM12y

BTW, I know it's not terribly rare for anti-marijuana laws to be enforced against middle-class people where I am; so he should have either specified “against middle-class people in Northern California” (but how is someone from (say) rural Poland supposed to know?) or use a different example such as copyright infringement for personal use (hoping that no country actually enforces that non-negligibly often). EDIT: A better criterion that would include laws against kidnapping but not laws against marijuana or laws against copyright infringement (though by far not a perfect one) in the context of ‘suggesting breaking those publicly on the internet would look bad’ would be ‘laws that a supermajority of internet users aged between 18 and 35 and with IQ above 115 would likely find ridiculous’. (Though I might be excessively Generalizing From One Example when thinking about what other people would think of anti-marijuana laws or copyright laws.)

6kodos9612y

Also, even in California, and even for people of middle class, you'll get marijuana laws enforced against you if you manage to piss off the wrong cop/prosecutor.

[-][anonymous]12y230

Just because I think responses to this post might not have been representative:

I think this is a good policy.

8Kaj_Sotala12y

I also agree with this policy, and feel that many of the raised or implied criticisms of it are mostly motivated from an emotional reaction against censorship. The points do have some merit, but their significance is vastly overstated. (Yes, explicit censorship of some topics does shift the Schelling fence somewhat, but suggesting that violence is such a slippery topic that next we'll be banning discussion about gun control and taxes? That's just being silly.)

8kodos9612y

You may think it's silly, others do not. Even if Eliezar has no intention of interpeting "violence" that way, how do we know that? Ambiguity about what is and is not allowed results in chilling far more speech than may have been originally intended by the policy author. Also, the policy is not limited to only violence, but to anything illegal (and commonly enforced on middle class people). What the hell does that even mean? Illegal according to whom? Under what jurisdiction? What about conflicts between state/federal/constitutional law? I mean, don't get me wrong, I think I have a pretty good idea what Eliezar meant by that, but I could well be wrong, and other people will likely have different ideas of what he meant. Again, ambiguity is what ends up chilling speech, far more broadly than the original policy author may have actually intended. And I will again reiterate what I consider to be the most slam-dunk argument against this policy: in the incident that provoked this policy change, the author of the offending post voluntarily removed it, after discussion convinced him it was a bad idea. Self-policing worked! So what exactly is the necessity for any new policy at all?

3Kaj_Sotala12y

I agree that your points about ambiguity have some merit, but I don't think there's much of a risk of free speech being chilled more than was intended, because there will be people who test these limits. Some of their posts will be deleted, some of them will not. And then people can see directly roughly where the intended line goes. The chilling effect of censorship would be a more worrying factor if the punishment for transgressing was harsher: but so far Eliezer has only indicated that at worst, he will have the offending post deleted. That's mild enough that plenty of people will have the courage to test the limits, as they tested the limits in the basilisk case. As for self-policing, well, it worked once. But we've already had trolls in the past, and the userbase of this site is notoriously contrarian, so you can't expect it to always work - if we could just rely on self-policing, we wouldn't need moderators in the first place.

0DanArmak12y

What about gasp whole other countries outside the US?

0kodos9612y

Yes, that was covered by the previous question: "Under what jurisdiction?"

[-]ChristianKl12y220

On of the most challenging moderation decisions I had to do at another forum was whether someone who argues the position "Homosexuality is a crime. In my country it's punishable with death. I like the laws of my country" should have his right of free speech. I think the author of the post was living in Uganda.

The basic question is, should someone who's been raised in Uganda feel free to share his moral views? Even if those views are offensive to Western ears and people might die based on those views?

If you want to have a open discussion about morality I think it's very valuable to have people who aren't raised in Western society participating openly in the discussion. I don't think LessWrong is supposed to be a place where someone from Uganda should be prevented from arguing the moral views in which he believes.

When it comes to politics, communists argue frequently for the necessarity of a revolution. A revolution is an illegal act that includes violence against real people. Moldburg argues frequently for the necessity of a coup d'état.

This policy allows for censoring both the political philosophy of communism as well as the political philosophy of moldbuggianism.
Even... (read more)

3A1987dM12y

Does it? CDT most certainly does, but...

2ChristianKl12y

Okay, you can argue whether it does. Regardles, that's an argument that should be possible in depth. And it should be possible to exchange the trolly cars for more real world examples.

0Eugine_Nier12y

No you're advocating changing the law. It's not a crime once/if the law is changed. Depends on where you are.

6ChristianKl12y

No, that sentence doesn't include the word law. It's a valid position to argue that a policeman has the moral duty to do everything he can to safe a life even when that involves breaking the law.

[-]DataPacRat12y220

I currently find myself tempted to write a new post for Discussion, on the general topic of "From a Bayesian/rationalist/winningest perspective, if there is a more-than-minuscule threat of political violence in your area, how should you go about figuring out the best course of action? What criteria should you apply? How do you figure out which group(s), if any, to try to support? How do you determine what the risk of political violence actually is? When the law says rebellion is illegal, that preparing to rebel is illegal, that discussing rebellion even in theory is illegal, when should you obey the law, and when shouldn't you? Which lessons from HPMoR might apply? What reference books on war, game-theory, and history are good to have read beforehand? In the extreme case... where do you draw the line between choosing to pull a trigger, or not?".

If it was simply a bad idea to have such a post, then I'd expect to take a karma hit from the downvotes, and take it as a lesson learned. However, I also find myself unsure whether or not such a post would pass the muster of the new deletionist criteria, and so I'm not sure whether or not I would be able to gather that idea - let a... (read more)

5MixedNuts12y

Do you have answers to those questions? Just "Hey, this problem exists" has not historically been shown to be productive.

8DataPacRat12y

I have /a/ set of answers, based on what I've learned so far of economics, politics, human nature, and various bits of evidence. However, I peg my confidence-levels of at least some of those answers as being low enough that I could be easily persuaded to change my mind, especially by the well-argued points that tend to crop up around here.

[-]CronoDAS12y220

The "interesting" thing about violence is that it's one of the few ways that a relatively small group of (politically) powerless people with no significant support can cause a big change in the world. However, the change rarely turns out the way the small group would hope; most attempts at political violence by individuals or small groups fail miserably at achieving the group's aims.

8BrassLion12y

Non-violent action has a reasonable track record, considering how rarely it's been used in an organized way by the oppressed. The track record is particularly good in the first world, where people care about appearances.

1[anonymous]12y

I can't think of any cases. Can you give some specific examples?

3BrassLion12y

Gandhi and Marting Luther King, Jr. are the headliners, as usual. Both used pacificism as a tool against regimes that, in the end, needed to think of themselves as decent people, and that had to bow to political pressure both at home and abroad. There's far more examples, though, that people don't think about - when you're looking for social change in the modern first world, non-violence is the default. Women's rights were secured without violence. Black civil rights in America were gained through non-violent activists like King and through the courts - there were violence groups like the Black Panthers, but in the end King's approach worked and violence... just didn't. Gay rights might be another example, although gays are marginalized, but not powerless, since they can show up anywhere - still, the gay rights movement has been well organized, never used violence, and has brought the first world to the point where full equality for homosexuals seems inevitable in about a generation.

6[anonymous]12y

Interesting. I've also seen analyses that argue that Gandhi and MLK were substantially helped by being backed by violent terrorist groups. Of course those analyses don't explain female and gay rights. I don't have the history and poli sci qualifications to judge the factors involved, but thanks for your take.

2JonathanLivengood12y

I'm not sure what you count as violence, but if you look at the history of the suffrage movement in Britain, you will find that while the movement started out as non-violent, it escalated to include campaigns of window-breaking, arson, and other destruction of property. (Women were also involved in many violent confrontations with police, but it looks like the police always initiated the violence. To what degree women responded in kind and whether that would make their movement violent is unclear to me.) The historians usually describe the vandalism campaigns as violent, militarism, or both, though maybe you meant to restrict attention to violence against persons. Of course, the women agitating for the vote suffered much more violence than they inflicted.

2MixedNuts12y

How violent is violence? Stonewall was a throw-bricks-type riot, but there were no assassinations or the like. Also there were some violent feminists, but as you say, Black Panthers.

0[anonymous]12y

Never say never. http://en.wikipedia.org/wiki/Stonewall_riots

[-]drethelin12y220

Got it. Posts discussing our plans for crimes will herewith be kept to the secret boards only.

8David_Gerard12y

And the mailing lists, apparently.

5Eliezer Yudkowsky12y

The Surgeon General recommends that you not discuss criminal activities, with respect to laws actually enforced, on any mailing list containing more than 5 people.

4[anonymous]12y

Why 5?

2Waffle_Iron12y

Have you ever tried to get a group of more than 5 people to keep a secret?

3[anonymous]12y

Have you ever tried to get a group of 4 people to keep a secret? I'm just wondering where the particular number comes from. Three people can keep a secret if two are dead and all that...

2David_Gerard12y

I was thinking of the London list, and this thread, about a drug which isn't actually illegal in the UK (it's prescription-restricted, but not illegal at all to possess) but selling it in public in a pub as if it is. I mean, WHAT. There's stupidity that isn't actually illegal but is nevertheless blithering.

0FiftyTwo12y

That seems sensible enough, you are allowed the drug if a competent expert has determined it is in your best interests to have it, but as you are not yourself qualifies to make that decision you can't transfer ownership to others.

0AndrewH12y

Intriguing, actual paraphrasing here of a US "The Surgeon General"? I can imagine it is something someone in high office might say.

9katydee12y

The Surgeon General is someone who issues national health recommendations. The implication of Eliezer's post is that discussing criminal activity may be hazardous to your health.

5Alicorn12y

We have a The Surgeon General, but he recommends things about smoking and whatnot; I'm pretty sure he doesn't issue warnings about mailing lists.

6timtyler12y

I believe the traditional structure is a clandestine cell system.

2Kawoomba12y

Back in line with you!

[-]pleeppleep12y210

Deleting comments for being perceived as dangerous might get in the way of conversation. I think that if we're worried about how the site looks to outsiders then it's probably only necessary to worry about actual posts. Nobody expects comments to be appropriate on the internet, so it probably doesn't hurt us that much.

0A1987dM12y

It was a top-level post (though one in Discussion) he was thinking about.

8pleeppleep12y

I know, but he said that the suggested policy change would include comments.

[-]A1987dM12y130

That's usual Yudkowskian overreaction he will likely get tired of implementing within a couple years or less.

7pleeppleep12y

....... But the site's only been around for a couple of years in the first place

2A1987dM12y

Well, when saying that, I was thinking of That Thing that happened in July 2010 about which EY appears to no longer have as many bees in his bonnet.

0MixedNuts12y

Does he appear so?

5A1987dM12y

In a couple of occasions in the last months I saw people discussing That Thing openly, and I myself encouraged them to rot13 stuff. And very recently there was a discussion which could be considered relevant to which I made non-rot13ed comments. Now maybe EY just didn't see those, but maybe he just didn't care too much.

[-][anonymous]12y210

Yes, a post of this type was just recently made.

Well then.

I've heard that firemen respond to everything not because they actually have to, but because it keeps the drill sharp, so to speak. The same idea may apply to mod action... (in other words, MOAR "POINTLESS" CENSORSHIP)

More seriously, does this policy apply to things like gwern's hypothetical bombing of intel?

2RomeoStevens12y

gwern specifically argued that small scale terrorism would be ineffective.

[-]printing-spoon12y170

Implying that whether his post should be censored hinges on the conclusion reached and not just the topic?

1RomeoStevens12y

discussion of violence by state actors is quite a bit different than discussion of individual violence.

5Jayson_Virissimo12y

Sure, but why is that a difference that makes a difference?

0MixedNuts12y

Individuals are somewhat likely to become violent because of Internet sophistry. If big oils (or likely future big oils) become violent because of Internet sophistry, we have bigger problems.

8TheOtherDave12y

I suppose the next question is whether it would apply to things like comments in response to gwern's hypothetical bombing of intel arguing that his conclusion is incorrect. Given the stated principles governing the new censorship policy, I think the answer would be "yes, of course."

6[anonymous]12y

Let's not delete posts for disagreeing on uncomfortable empirical questions.

2TheOtherDave12y

I don't think the policy EY is proposing involves banning people, just deleting the stuff we write that violates policy.

2[anonymous]12y

fixed, thanks

0timtyler12y

It looks as though that was on gwern.net - outside the zone.

5[anonymous]12y

it was in discussion too.

-3Epiphany12y

If you're talking about his Slowing Moore's Law: Why You Might Want To and How You Would Do It it's not there anymore. I didn't thoroughly read the new version on his site, so there's a chance that there is now a link to an article that will still be confused for a pro-terrorism piece (that's the problem the previous version had) or sounds like it's advocating the idea of governments attacking chip fabs.

1gwern12y

I posted a draft here. A while after the initial discussion, because I had expanded it massively, I deleted the draft version so readers of that post had no choice but to go to the updated master copy. (I also did this for all similar posts like my Melatonin post, for similar reasons.)

-2MugaSofer12y

As worded, yes. However, I suspect it wouldn't have been enforced in that case.

[-][anonymous]12y190

violence against real people.

Abortion, euthenasia and suicide fit that description, some say. For them and those who disagree with them this proposal may have unforeseen consequences. Edit: all three are illegal in parts of the world today.

[-]BrassLion12y180

I think this is an overreation to (deleted thing) happening, and the proposed policy goes too far. (Deleted thing) was neither a good idea or good to talk about in this public forum, but it was straight-out advocating violence in an obvious and direct way, against specific, real people that aren't in some hated group. That's not okay and it's not good for community for the reasons you (EY) said. But the proposed standard is too loose and it's going to have a chilling effect on some fringe discussion that's probably going to be useful in teasing out some of the consquences of ethics (which is where this stuff comes up). Having this be a guideline rather than a hard rule seems good, but it still seems like we're scarring on the first cut, as it were.

I think we run the risk of adopting a censorship policy that makes it difficult to talk about or change the censorship policy, which is also a really terrible idea.

I agree with the general idea of protecting LW's reputation to outsiders. After all, if we're raising the sanity waterline (rather than researching FAI), we want outsiders to become insiders, which they won't do if they think we're crazy.

"No advocating violence agains... (read more)

[-]Nominull12y180

Censorship is particularly harmful to the project of rationality, because it encourages hypocrisy and the thinking of thoughts for reasons other than that they are true. You must do what you feel is right, of course, and I don't know what the post you're referring to was about, but I don't trust you to be responding to some actual problematic post instead of self-righteously overreacting. Which is a problem in and of itself.

8kodos9612y

Passive-aggression level: Obi-Wan Kenobi

3gjm12y

I don't see that that's passive-aggressive when it's accompanied by a clear and explicit statement that Nominull thinks Eliezer is wrong and why. What would be passive-aggressive is just saying "Well, I suppose you must do what you feel is right" and expecting Eliezer to work out that disapproval is being expressed and what sort.

0kodos9612y

I didn't mean it as a criticism, just that my brain pattern-matched his choice of words and read it in Alec Guiness's voice.

2twanvl12y

In particular, this comment seems to suggest that EY considers public opinion to be more important than truth. Of course this is a really tough trade-off to make. Do you want to see the truth no matter what impact it has on the world? But I think this policy vastly overestimates the negative effect posts on abstract violence have. First of all, the people who read LW are hopefully rational enough not to run out and commit violence based on a blog post. Secondly, there is plenty of more concrete violence on the internet, and that doesn't seem to have to many bad direct consequences.

3Viliam_Bur12y

Anyone can read LW. There is no IQ test, rationality test, or a mandatory de-biasing session before reading the articles and discussions. I am not concerned about someone reading LW and commiting a violence. I am concerned about someone commiting violence and coincidentally having read LW a day before (for example just one article randomly found by google), and police collecting a list of recently visited websites, and a journalist looking at that list and then looking at some articles on LW. Shortly, we don't live on a planet full of rationalists. It is a fact of life that anything we do can be judged by any irrational person who notices. Sure, we can't make everyone happy. But we should avoid some things that can predictably lead to unnecessary trouble.

[-]Pentashagon12y170

Do wars count? I find it strange, to say the least, that humans have strong feelings about singling out an individual for violence but give relatively little thought to dropping bombs on hundreds or thousands of nameless, faceless humans.

Context matters, and trying to describe an ethical situation in enough detail to arrive at a meaningful answer may indirectly identify the participants. Should there at least be an exception for notorious people or groups who happen to still be living instead of relegated to historical "bad guys" who are almost universally accepted to be worth killing? I can think of numerous examples, living and dead, who were or are the target of state-sponsored violence, some with fairly good reason.

[-]MixedNuts12y170

Your generalization is averaging over clairvoyance. The whole purpose of discussing such plans is to reduce uncertainty over their utility; you haven't proven that the utility gain of a plan turning out to be good must be less than the cost of discussing it in public.

Does the policy apply to violence against oneself? (I'm guessing not, since it's not illegal.) Talking about it is usually believed to reduce risk.

There's a scarcity effect whereby people believe pro-violence arguments to be stronger, since if they weren't convincing they wouldn't be censored. Not sure how strong it is, likely depends on whether people drop the topic or say things like "I'm not allowed to give more detail, wink wink nudge nudge".

It's a common policy so there don't seem to be any slippery slope problems.

We're losing Graham cred by being unwilling to discuss things that make us look bad. Probably a good thing, we're getting more mainstream.

1RomeoStevens12y

since when is violence against oneself or even discussion of violence against oneself fully legal?

5wedrifid12y

In most times and places throughout history, including all countries whose legal systems I am familiar with.

3Caspian12y

Suicide in particular is often illegal. ETA: possibly this statement of mine was outdated.

2wedrifid12y

Either you or some of the people reading your comment seem to have been mislead into concluding that a thing being illegal and also violence against oneself can be generalised to conclude that violence against oneself or even discussion of violence against oneself is illegal. That seems to be a rather blatant confusion.

0kodos9612y

I'm not sure what RomeoStevens meant about discussion of violence against oneself being illegal, but aside from that aspect, his point is entirely valid. You seem to be suggesting that we're generalising from "suicide is illegal" to "any form of violence against oneself is illegal". We're not. We're simply noting that suicide is one type of violence against onself, and it's illegal. Your statement expands to "In most times and places throughout history, including all countries whose legal systems I am familiar with, violence against oneself is fully legal." Unless you're familiar only with very odd legal systems, that seems to be a rather blatant confusion.

-2wedrifid12y

No. MixedNut's point. RomeoStevens' reply was confused and mistaken. Unfortunately Caspian has mislead you about the context. That was my original impression and why I refrained from downvoting him. Until, that is, it became apparent that he and some readers (evidently yourself included) believe that his statement of trivia in some way undermines the point made by MixedNut's and supported by myself or supports RomeoStevens' ungrammatical rhetorical interjection.

-2kodos9612y

I had read the entire context, and re-read it just now to make sure I hadn't missed anything. You're correct that RomeoStevens' reply doesn't really undermine MixedNuts' point, and is therefore "trivia". But it's nonetheless correct trivia (modulo the above-mentioned caveat) and your refutation of it is therefore quite confusing. But it's pointless to continue arguing this trivial point, as it's irrelevant to the thread topic, except in the meta sense that these kinds of pointless semantic debates will be the inevitible result of implementation of this extremely ill-advised and poorly thought-through censorship policy.

1MixedNuts12y

What are you thinking of? Non-assisted suicide that doesn't put third parties in danger is legal most places (exceptions: India, Singapore, North Korea, Virginia). Self-injury is legal in the US at least. Discussion of suicide is allowed as long as it's even slightly more hypothetical than "I intend to kill myself in the near future". Discussion of self-injury is AFAIK completely legal (in the US?).

2RomeoStevens12y

My understanding has always been that self harm or plausible discussion of self harm in the US leads to a loss of autonomy in that you can be diagnosed with a mental illness and lose access to things like voting, driving, firearms, etc. (depending on the diagnosis)

2MixedNuts12y

Trigger warning for, obviously, self-harm. There's a huge chasm between a mental illness diagnosis (which self-harm is very likely to cause, especially in the US where you need diagnosis other than "ain't quite right - not otherwise specified" for insurance) and actual repercussions. Members of online support groups report that their psychiatrists either treat self-injury like any other symptom (asking about it, describing decreases as good but not praiseworthy) or recommend they stop but do not enforce it. If it gets life-threatening it's treated like suicide, but that almost never comes up.

0DanArmak12y

What does it mean to make suicide illegal, anyway? You can't punish the perpetrator, they're dead. You can punish their relatives by e.g. taking away their inheritance, but someone who plans their suicide in advance can circumvent that by transferring ownership of the important things before killing themselves.

1MixedNuts12y

Punish attempts. Punish in ways that are avoidable (e.g. inheritance) but work for insufficiently planned suicides. If there's a state religion, predict punishment in the afterlife. Punish relatives directly (North Korea does that).

1Decius12y

It means that you prosecute failed suicides as crimes.

0DanArmak12y

Is there good data on whether this is effective as deterrence? I don't expect it could be effective as punishment: I would expect it to increase despair and poverty, and so to increase chances of recurrent attempts.

0DanArmak12y

I'm not sure if that's a good thing...

0MixedNuts12y

Once you have so many smart contrarians that you run into sharp diminishing returns trying to recruit more, you want to attract smart non-contrarians. To pick a very silly example, a group of mostly Gentiles musing aimlessly on the ethics of genociding Jews (because it's a local point of pride to play with any idea no matter how evil or stupid) is going to have a hard time attracting Jews.

1DanArmak12y

Why are there diminishing returns? Because too many smart contrarians cannot coexist? Because we ran out of smart contrarians to recruit? Because a group requires non-smart or non-contrarian people too in order to function better? Also: over the last year many people joined LW, many of them referred here by HPMOR. I would expect these people to be less smart-contrarian.

[-][anonymous]12y160

Would pro-suicide and general anti-natalist posts be covered by this?

3Viliam_Bur12y

Suggesting that specific people commit suicide, obviously yes. People in general... maybe no. I am not going to explain why, but although death of all people technically includes the death of any specific person X.Y., saying "X.Y. should die" sounds worse than saying "all humans should die".

0eurg12y

Forget about it.

5[anonymous]12y

I'm not trolling. I quite like reading Sister Y's stuff and have said so in the past.

0eurg12y

Luckily enough, that blog seems much better than your introduction of it. My troll accusation is founded on your highly repetitive deliberate misunderstanding of the OP. It must be deliberate, as you are usually much smarter than that, and also better in style. Also, Sister Y is not pro-suicide per se, but against anti-suicide positions; at least that's how I read it.

-1MugaSofer12y

Um, yes. We don't ant to look like a suicide phyg.

[-]Eugine_Nier12y160

I don't necessarily object to this policy but find it troubling that you can't give a better reason for not discussing violence being a good idea than PR.

Frankly, I find it even more troubling that your standard reasons for why violence is not in fact a good idea seem to be "it's bad PR" and "even if it is we shouldn't say so in public".

As I quote here:

if your main goal is to show that your heart is in the right place, then your heart is not in the right place.

Edit: added link to an example of SIAI people unable to give a better reason against doing violence than PR.

[-]quiet12y110

I appreciate the honesty of it. No one here is going to enact any of these thought experiments in real life. The likely worst outcome is to off-put potential SI donors. It must be hard enough to secure funding for a fanfic-writing apocalypse cult; prepending violent onto that description isn't going to loosen up many wallets.

6jimrandomh12y

I would find this troubling if it were true, but the better reason is right there in the post: "Talking about such violence makes that violence more probable".

0Desrtopa12y

If the violence is a bad idea, which in nearly all cases it probably would be, other commenters are likely to point that out. Having people inspired to carry out acts of violence in spite of other members pointing out that it's unlikely to bear good results is possible, but unlikely, whereas having people judge the community negatively for discussing such things at all is considerably more likely.

1Eugine_Nier12y

Can you point to an example of this actually happening?

4Desrtopa12y

Here.

1DanArmak12y

The post in question was heavily downvoted before it was deleted.

[-]MixedNuts12y150

The freaky consequences are not of the policy, they're of the meta-policy. You know how communities die when they stop being fun? Occasional shitstorms are not fun, and fear of saying something that will cause a shitstorm is not fun. Benevolent dictators work well to keep communities fun; the justifications don't apply when the dictator is pursuing goals that aren't in the selfish interest of members and interested lurkers; making the institute the founder likes look bad only weakly impacts community fun.

Predictable consequences are bright iconoclasts leaving, and shitstorm frequency increasing. (That's kinda hard to settle: the former is imprecise and the latter can be rigged.)

Every time, people complain much less about the policy than about not being consulted. There are at least two metapolicies that avoid this:

Avoid kicking up shitstorms. In this particular instance, you could have told CronoDAS his post was stupid and suggest he delete it, and then said "Hey, everyone, let's stop talking about violence against specific people, it's stupid and makes us look bad" without putting your moderator hat on.
Produce a policy, possibly ridiculously stringent, that covers most things you don't like, which allows people to predict moderator behavior and doesn't change often. Ignore complaints when enforcing, and do what you wish with complaints on principle.

-6drethelin12y

[-]shware12y150

Taking this post in the way it was intended i.e. 'are there any reasons why such a policy would make people more likely to attribute violent intent to LW' I can think of one:

The fact that this policy is seen as necessary could imply that LW has a particular problem with members advocating violence. Basically, I could envision the one as saying: 'LW members advocate violence so often that they had to institute a specific policy just to avoid looking bad to the outside world'

And, of course, statements like 'if a proposed conspiratorial crime were in fact good you shouldn't talk about it on the internet' make for good out-of-context excerpts.

3fubarobfusco12y

See also xkcd.

1ChristianKl12y

I don't think that's probable. There are many online forum that have forum rules that prevent those discussions.

[-][anonymous]12y150

Posts or comments advocating or 'asking about' violation of laws that are actually enforced against middle-class people (e.g., kidnapping, not anti-marijuana laws) may at the admins' option be censored

The blasphemy laws of many countries fit this description - another possible unintended consequence.

[-]Eugine_Nier12y150

I find that threatening hypothetical violence against my interlocutor can be a useful rhetorical device for getting them to think about ethical problems in near mode.

[-]FiftyTwo12y500

I'm going to hit you with a stick unless you can give me an example of where that has been effective.

9Pentashagon12y

THREE examples.

5kodos9612y

For all the whining I do about how LWers lack a sense of humor.... I absolutely love it when I'm proven wrong.

[-]Qiaochu_Yuan12y100

Do you really feel like LWers lack a sense of humor? LWers have posted some of the funniest things I've ever read. Their sense-of-humor distribution has heavy tails, at least.

2kodos9612y

Yeah, I'd say that's a fair assesment.

[-]CronoDAS12y140

Maybe something like "Moderators, at their discretion, may remove comments that can be construed as advocating illegal activity" would work for a formal policy - it reads like relatively inoffensive boilerplate and would be something to point to if a post like mine needs to go, but is vague enough that it doesn't scream "CENSORSHIP!!!" to people who feel strongly about it. The "at their discretion" is key; it doesn't create a category of posts that moderators are required to delete, so it can't be used by non-moderators as a weapon to stifle otherwise productive discussion. (If you don't trust the discretion of the moderators, that's not a problem that can be easily solved with a few written policies.)

[-]asparisi12y140

Yeesh. Step out for a couple days to work on your bodyhacking and there's a trench war going on when you get back...

In all seriousness, there seems to be a lot of shouting here. Intelligent shouting, mind you, but I am not sure how much of it is actually informative.

This looks like a pretty simple situation to run a cost/benefit on: will censoring of the sort proposed help, hurt, or have little appreciable effect on the community.

Benefits: May help public image. (Sub-benefits: Make LW more friendly to new persons, advance SIAI-related PR); May reduce brain-eating discussions (If I advocate violence against group X, even as a hypothetical, and you are a member of said group, then you have a vested political interest whether or not my initial idea was good which leads to worse discussion); May preserve what is essentially a community norm now (as many have noted) in the face of future change; Will remove one particularly noxious and bad-PR generating avenue for trolling. (Which won't remove trolling, of course. In fact, fighting trolls gives them attention, which they like: see Cons)

Costs: May increase bad PR for censoring (Rare in my experience, provided that the rules are sensibly ... (read more)

[-]kodos9612y140

Aside from the fact that "it might make us look bad" is a horrible argument in general, have you not considered the consequence that censorship makes us look bad? And consider the following comment below:

Got it. Posts discussing our plans for crimes will herewith be kept to the secret boards only.

It was obviously intended as a joke, but is that clear to outsiders? Does forcing certain kinds of discussions into side-channels, which will inevitibly leak, make us look good?

Consideration of these kinds of meta-consequences is what separates naive decision theories from sophisticated decision theores. Have you considered that it might hurt your credibility as a decision theorist to demonstrate such a lack of application of sophisticated decision theory in setting policies on your own website?

And now, what I consider to be the single most damning argument against this policy: in the very incident that provoked this rule change, the author of the post in question, after discussion, voluntarily withdrew the post, without this policy being in effect! So self-policing has demonstrated itself, so far, to be 100% effective at dealing with this situation. So where exactly is the necessity for such a policy change?

0handoflixue12y

You can argue that LessWrong shouldn't care about PR, or that censorship is going to be bad PR, or that censorship is unnecessary, but you can't argue that PR is a fundamentally horrible idea without some very strong evidence (which you did not provide). - It's almost tautological that if a group cares about PR, it HAS to care about what makes them look bad: If Obama went on record saying that we should kill everyone on Less Wrong, and made it clear he was serious, I'd hope to high hell that there would be an impeachment trial. If Greenpeace said we should kill all the oil CEOs, people would consider them insane terrorists. If the oil CEOs suggested that there might be... incentives... should Greenpeace members be killed...

-1kodos9612y

That was perhaps a bit of an overstatement on my part. Considering PR consequences of actions is certainly a good thing to do. But if PR concerns are driving your policy, rather than simply informing it, that's bad.

-1handoflixue12y

Taboo "driving" and "informing" and explain the difference between those two to me? Or we can save ourselves some time if this resolves your objection: Eliezer is saying that he is adding the OPTION to censor things if they are a PR problem OR because the person is needlessly incriminating themselves. I'm not sure how that's a bad OPTION to have, given that he's explicitly stated he will not mindlessly enforce it, and in fact has currently enforced it zero (0) times to my knowledge (the post that prompted this was voluntarily withdrawn by it's author)

3kodos9612y

One the one hand, you're deciding policy based on non-PR related factors, then thinking about the most PR friendly way to proceed from there. On the other hand, you're letting PR actually determine policy.

1handoflixue12y

Which category is it if you decide based on multiple factors, ONE of which is PR? And why is this a bad thing, if that's what you believe?

0kodos9612y

Before I spend any more time replying to this, can you clarify for me... do you and I actually disagree about something of substance here? I.e. how an organization should, in the real world, deal with PR concerns? Or are we just arguing about the most technically correct way to go about stating our position?

[-][anonymous]12y130

(i.e., even if a proposed conspiratorial crime were in fact good, there would still be net negative expected utility from talking about it on the Internet; if it's a bad idea, promoting it conceptually by discussing it is also a bad idea; therefore and in full generality this is a low-value form of discussion).

This seems to be a fully general argument against Devil's Advocacy. Was it meant as such?

0Eliezer Yudkowsky12y

I don't see the link, but it does so happen I think "devil's advocate!" is mentally poisonous. I'd even call it an evolutionary precursor of trolling. http://lesswrong.com/lw/r3/against_devils_advocacy/

1[anonymous]12y

I recall reading this before but reread and did some thought on it again. We may have a sort of a trade off here. I think I agree on Devil's Advocacy might be risky for the person doing it but I see obvious benefits in having one or two in a group as it emulates some of the epistemic benefits of value diversity as well as signals to the group that its beliefs shouldn't be taken that seriously. This is the major problem I see with my view that I'm unsure how to resolve though. Devil's Advocates may in practice be merely straw men generators. Interesting I suppose I agree with it.

0buybuydandavis12y

You may only generate straw men while you attempt to generate steel men, but how many steel men are you likely to make if you don't even try to make them? You can always fail, but not trying guarantees failure.

0MugaSofer12y

Sounds more like a precursor of "Policy Debates Should Not Appear One-Sided".

[-]Joshua Hobbes12y130

Would this censor posts about robbing banks and then donating the proceeds to charity?

[-]Eliezer Yudkowsky12y310

Depends on exactly how it was written, I think. "The paradigmatic criticism of utilitarianism has always been that we shouldn't rob banks and donate the proceeds to charity" - sure, that's not actually going to conceptually promote the crime and thereby make it more probable, or make LW look bad. "There's this bank in Missouri that looks really easy to rob" - no.

[-][anonymous]12y200

Uncharitable reading: As long as taking utilitarianism seriously doesn't lead to arguments to violate formalized 21st century Western norms too much it is ok to argue for taking utilitarianism seriously. You are however free to debunk how it supposedly leads to things considered unacceptable on the Berkeley campus in 2012, since it obviously can't.

[-][anonymous]12y110

What abot pro-robbing banks in general?

5Viliam_Bur12y

Best way would be to construct the comment in a way that makes it least likely to seem bad when quoted outside of LW. For example we could imagine an alternative universe with intelligent bunnies and carrot-banks. Would it be good if a bunny robbed the carrot-bank and donated the carrots to charity? If someone copied this comment on a different forum, it would seem silly, but not threatening. It is more difficult to start a wave of moral panic because of carrots and bunnies.

4Decius12y

What about discussions which discuss flaws in security systems, generally? e.g. "Banks often have this specific flaw which can be mitigated in this cost-ineffective manner."?

[-]Alicorn12y130

Or Really Extreme Altruism?

3Larks12y

Note to all: Alicorn is referring to something else. Robbing banks may be extreme but it is not altruism.

1Alicorn12y

Edited in a link.

0Eliezer Yudkowsky12y

This does indeed seem like something that's covered by the new policy. It's illegal. In the alternative where it's a bad idea, talking about it has net negative expected utility. If it were for some reason a good idea, it would still be incredibly stupid to talk about it on the &^%$ing Internet. I shall mark it for deletion if the policy passes.

[-]Tenoke12y130

So you don't see value in discussions like these? Thought experiments that give some insights into morality? Is really the (probably barely any) effect on the reputation of LW because of those posts really more than the benefit of the discussion?

2Eliezer Yudkowsky12y

I think that post was a net negative effect on reality and that diminishing the number of people who read it again is a net positive. No, the conversation isn't worth it.

3Tenoke12y

Oh come on, you are evoking your basilisk-related logic here? How does it have a negative effect. Please don't tell me that it is because you think that there will be more suicides in the world if the number of readers of the post is larger? And further please don't tell me that if you thought that you think that this will lead to a net negative effect for the world? But please do answer me.

[-]Eliezer Yudkowsky12y130

It has a net negative effect because people then go around saying (this post will be deleted after policy implementation), "Oh, look, LW is encouraging people to commit suicide and donate the money to them." That is what actually happens. It is the only real significant consequence.

Now it's true that, in general, any particular post may have only a small effect in this direction, because, for example, idiots repeatedly make up crap about how SIAI's ideas should encourage violence against AI researchers, even though none of us have ever raised it even as a hypothetical, and so themselves become the ones who conceptually promote violence. But it would be nice to have a nice clear policy in place we can point to and say, "An issue like this would not be discussable on LW because we think that talking about violence against individuals can conceptually promote such violence, even in the form of hypotheticals, and that any such individuals would justly have a right to complain. We of course assume that you will continue to discuss violence against AI researchers on your own blog, since you care more about making us look bad and posturing your concern, than about the fact that you, yourself, are the one has actually invented, introduced, talked about, and given publicity to, the idea of violence against AI researchers. But everyone else should be advised that any such 'hypothetical' would have been deleted from LW in accordance with our anti-discussing-hypothetical-violence-against-identifiable-actual-people policy."

[-]fubarobfusco12y170

idiots repeatedly make up crap

Idiots make up crap. You probably can't change this. The more significant you are, the more crap idiots will make up about you. Idiots claim that Barack Obama is a Kenyan Muslim terrorist and that George Bush is mentally subnormal. Not because they have sufficient evidence of these propositions, but because gossip about Obama and Bush is thereby juicier than gossip about my neighbor Marty whom you've never heard of.

Idiots make up crap about projects, too. They say NASA faked the moon landing, vaccines cause autism, and that international food aid contains sterility drugs. It turns out that scurrilous rumors about NASA and the United Nations spread farther than scurrilous rumors about that funny-looking building in the town park which is totally a secret drug lab for the mayor.

But everyone else should be advised that any such 'hypothetical' would have been deleted from LW in accordance with our anti-discussing-hypothetical-violence-against-identifiable-actual-people policy."

How about treating the hypothetical as the stupidity it is? "Dude, beating up AI researchers wouldn't work and you're a jerk for posting it. There are a half dozen obvious reasons it wouldn't work, if you take five minutes to think about it ... and you're a jerk for posting it because it's stirring up shit for no good reason. Seriously, quit it. This is LW, not Conspiracy Hotline."

9Eugine_Nier12y

And yet, when attempting to list them, the only one anyone from SIAI can seem think of is bad PR.

2Viliam_Bur12y

The important difference is that in these cases the given idiot is less famous than the person they make crap about. Imagine an alternative universe where Barack Obama is just an unknown guy, and some idiots for whatever reason start claiming that he is a Muslim terrorist. I can imagine an anonymous phone call to the police, a police action with some misunderstanding, resulting with too many negative utilons for Mr. Obama. In our universe, Mr. Obama has the advantage of being more famous than any such possible accuser. However, SIAI does not have the same advantage.

2Eliezer Yudkowsky12y

Sounds like a fine reply on LW. I think it will be useful, on forums not LW, to have a LW-policy to point to.

[-]kodos9612y170

It has a net negative effect because people then go around saying (this post will be deleted after policy implementation), "Oh, look, LW is encouraging people to commit suicide and donate the money to them." That is what actually happens. It is the only real significant consequence.

This is where the rubber meets the road as far as whether we really mean it when we say "that which can be destroyed by the truth, should be." If we accept this argument, then by "mere addition" of censorship rules, you eventually end up renaming SIAI "The Institute for Puppies and Unicorn Farts", and completly lying to the public about what it is you're actually about, in order to benefit PR.

[-]Eugine_Nier12y100

"Oh, look, LW is encouraging people to commit suicide and donate the money to them."

Well, are you?

idiots repeatedly make up crap about how SIAI's ideas should encourage violence against AI researchers, even though none of us have ever raised it even as a hypothetical,

True, but you have said things that seem to imply it. Seriously, you can't go around saying "X" and "X->Y" and then object when people start attributing position "Y" to you.

3Eliezer Yudkowsky12y

No. To prove this, I shall shortly delete the post advocating it. Point one: We never said X->Y. We said X, and a bunch of people too stupid to understand the fallacy of appeal to consequences said 'X->violence, look what those bad people advocate' as an attempted counterargument. Since no actual good can possibly come of discussing this on any set of assumptions, it would be nice to have the counter-counterargument, "Unlike this bad person here, we have a policy of deleting posts which claim Q->specific-violence even if the post claims not to believe in Q because the identifiable target would have a reasonable complaint of being threatened".

[-]kodos9612y210

it would be nice to have the counter-counterargument, "Unlike this bad person here, we have a policy of deleting posts which claim Q->specific-violence even if the post claims not to believe in Q because the identifiable target would have a reasonable complaint of being threatened".

I would find this counter-counter-argument extremely uncompelling if made by an opponent. Suppose you read someone's blog who made statements which could be interpreted as vaguely anti-Semitic, but it could go either way. Now suppose someone in the comments of that blog post replied by saying "Yeah, you're totally right, we should kill all the Jews!".

Which type of response from the blog owner do you think would be more likely to convince you that he was not actually an anti-Semite: 1) deleting the comment, covering up its existence, and never speaking of it, or 2) Leaving the comment in place, and refuting it - carefully laying out why the commenter is wrong.

I know that I for one would find the latter response much more convincing of the author's benign intent.

Note: in order to post this comment, despite it being, IMHO entirely on-point and important to the conversation, I had to take a 5 point karma hit.... due to the LAST poorly thought out, dictatorially imposed, consensus-defying policy change.

3Viliam_Bur12y

If someone really wants to get some cheap internet points for accusing the author of antisemitism, either option can be used. In both cases, the fact that the comment was written on the blog would be interpreted as an evidence for blog somehow evoking this kind of comment. Both deleting and refuting would be interpreted like "the author pretends to disagree, for obvious PR reasons, but he cannot fool us". The advantage of deleting the comment is that a potential accuser has smaller chance to notice it (well, unless some readers make "why did this specific comment disappear?" their topic of the month), and they cannot support their attacks with hyperlinks and screenshots. Also, if someone puts specific keywords in Google, they will not get that blog among results.

5CronoDAS12y

I wasn't thinking of SIAI as the charity.

[-]AdeleneDawner12y150

Regardless of your intentions, I know of one person who somewhat seriously considered that course of action as a result of the post in question. (The individual in question has been talked out of it in the short term, by way of 'the negative publicity would hurt more than the money would help', but my impression is that the chance that they'll try something like that has still increased, probably permanently.)

9CronoDAS12y

[sincerity mode]So... is that a good thing, or a bad thing?[/sincerity mode] In many circumstances, sacrificing one's own life in order to save others is considered a good thing, and people who do it are called "heroes". A famous example is the story of railroad engineer Casey Jones, who, after realizing that a collision with a stalled train was inevitable, chose to remain in the engine and slow his own train as much as possible, saving the rest of the passengers and crew at the cost of his own life. "Really Extreme Altruism" (with the money going to one of GiveWell's top charities) isn't as dramatic as a "typical" real-life Heroic Sacrifice, but the outcome is the same: one person dies, a lot of other people live who would have otherwise died. It's the manner of the sacrifice (and the distributed, distant nature of the benefit) that makes it far more disturbing.

0spzx11y

There should be a warning on the donate page: "For reasons of public relations, please refrain from donating and minimize your association with us if you are or may in the future become suicidal." Of course, if I were, not being able to contribute would be one less reason to stick around. I could shop for some less controversial group to support (possibly one that indirectly helped SIAI/MIRI), but it wouldn't be quite as motivating or as obviously sufficient to offset the cost of living.

[-]Eliezer Yudkowsky12y120

This intention of yours is not transparent. Plus, they don't care.

4CronoDAS12y

I edited the original post to link to GiveWell's top charities list.

3Tenoke12y

I thought I posted this comment last night, but it seems like I didn't (and now I have to pay karma to post it) but aren't we just encouraging belief bias this way? (which has an additional negative utility on top of the loss of positive utility from the discussion and loss of utility because people see us as a heavily-censored community and form another type of negative opinion of us)

2DanArmak12y

Idiots make up crap about all kinds of things, not just violence or other illegal acts. Ideas outside societal norms often attract bad PR. If your primary goal here is to improve PR, you would have to censor posts by explicit PR criteria. The proposed criteria of discussion of violence or law-breaking is not optimized for this goal. So, what is it you really want? Discussion of violence is something that (you claim) has no positive value, even ignoring PR. So it's easy to decide to censor it. But have you really considered what else to censor according to your goals? Violence clearly came due to the now deleted post; it was an available example. But you shouldn't just react to it and ignore other things, if your goal is not to prevent discussion of violence or crime in itself.

8CronoDAS12y

As far as I can tell, Really Extreme Altruism actually is legal.

4saturn12y

What about the possibility that someone who thought it was a good idea would change their mind after talking about it?

3Eliezer Yudkowsky12y

This seems an order-of-magnitude less likely than somebody wouldn't naturally think of the dumb idea, seeing the dumb idea.

8Decius12y

Therefore censor uncommon bad ideas generally?

-3wedrifid12y

This is an example of why I support this kind of censorship. Lesswrong just isn't capable of thinking about such things in a sane way anyhow. The top comment in that thread demonstrates AnnaSalamon being either completely and utterly mindkilled or blatantly lying about simple epistemic facts for the purpose of public relations. I don't want to see the (now) Executive Director of CFAR doing either of those things. And most others are similarly mindkilled, meaning that I just don't expect any useful or sane discussion to occur on sensitive subjects like this. (ie. I consider this censorship about as intrusive as forbidding peanuts to someone with a peanut allergy.)

[-]fubarobfusco12y180

The top comment in that thread demonstrates AnnaSalamon being either completely and utterly mindkilled or blatantly lying

This seems an excessively hostile and presumptuous way to state that you disagree with Anna's conclusion.

2wedrifid12y

No it isn't, the meaning of my words are clear and quite simply do not mean what you say I am trying to say. The disagreement with the claims of the linked comment is obviously implied as a premise somewhere in the background but the reason I support this policy really is because it produces mindkilled responses and near-obligatory dishonesty. I don't want to see bullshit on lesswrong. The things Eliezer plans to censor consistently encourage people to speak bullshit. Therefore, I support the censorship. Not complicated. You may claim that it is rude or otherwise deprecated-by-fubarobfusco but if you say that my point is different to both what I intended and what the words could possibly mean then you're wrong.

[-]fubarobfusco12y100

No it isn't, the meaning of my words are clear and quite simply do not mean what you say I am trying to say.

Well, taking your words seriously, you are claiming to be a Legilimens. Since you are not, maybe you are not as clear as you think you are.

It sure looks from what you wrote that you drew an inference from "Anna does not agree with me" to "Anna is running broken or disreputable inference rules, or is lying out of self-interest" without considering alternate hypotheses.

6jsalvatier12y

This also seems like an excessively hostile way of disagreeing! I think there's some illusion of transparency going on. I think Might have worked better

-7wedrifid12y

-1Pentashagon12y

There are a lot of topics about which most people have only bullshit to say. The solution is to downvote bullshit instead of censoring potentially important topics. If not enough people can detect bullshit that's an entirely different (and far worse) problem.

6Eugine_Nier12y

Yes and if the CFAR Executive Director is either mindkilled or willing to lie for PR, I want to know about it.

2jbeshir12y

I think that a discussion in which only most people are mindkilled can still be a fairly productive one on these questions in the LW format. LW is actually one of the few places where you would get some people who aren't mindkilled, so I think it is actually good that it achieves this much. They seem fairly ancillary tor LW as a place for improving instrumental or epistemic rationality, though. If you think testing the extreme cases of your models of your own decision-making is likely to result in practical improvements in your thinking, or just want to test yourself on difficult questions, these things seem like they might be a bit helpful, but I'm comfortable with them being censored as a side effect of a policy with useful effects.

2wedrifid12y

Unfortunately the non mindkilled people would also have to be comfortable simply ignoring all the mindkilled people so that they can talk among themselves and build the conversation toward improved understanding. That isn't something I see often. More often the efforts of the sane people are squandered trying to beat back the tide of crazy.

[-]kodos9612y120

I am asking in advance if anyone has non-obvious consequences they want to point out or policy considerations they would like to raise. In other words, the form of this discussion is not 'Do you like this?' - you probably have a different cost function from people who are held responsible for how LW looks as a whole - but rather, 'Are there any predictable consequences we didn't think of that you would like to point out

Eliezer, at this point I think it's fair to ask: has anything anyone has said so far caused you to update? If not, why not?

I realize some of my replies to you in this thread have been rather harsh, so perhaps I should take this opportunity to clarify: I consider myself a big fan of yours. I think you're a brilliant guy, and I agree with you on just about everything regarding FAI, x-risk, SIAI's mission.... I think you're probably mankind's best bet if we want to successfully navigate the singularity. But at the same time, I also think you can demonstrate some remarkably poor judgement from time to time... hey, we're all running on corrupted hardware after all. It's the combination of these two facts that really bothers me.

I don't know of any way to say this that ... (read more)

[-]CronoDAS12y120

I don't know if we actually need a specific policy on this. We didn't in the case of my post...

3DanArmak12y

I agree. We should trust in the community more where the guarantee of moderation (by establishing a policy) is not needed. Your post was quickly downvoted, and you deleted it yourself. This is an example of a good outcome that demonstrates we didn't need moderation.

0Viliam_Bur12y

On the other hand, if we will need it for some post in future, it will be an advantage that the policy will already have existed and will not have to be invented ad hoc.

[-]SoftFlare12y100

Beware Evaporative Cooling of Group Beliefs.

I am for the policy, although heavy-heartedly. I feel that one of the pillars of Rationality is that there should be no Stop Signs and this policy might produce some. On the other hand, I think PR is important, and that we must be aware of evaporative cooling that might happen if it is not applied.

On a neutral note - We aren't enemies here. We all have very similar utility functions, with slightly different weights on certain terminal values (PR) - which is understandable as some of us have more or less to lose from LW's PR.

To convince Eliezer - you must show him a model of the world given the policy that causes ill effects he finds worse than the positive effects of enacting the policy. If you just tell him "Your policy is flawed due to ambiguitiy in description" or "You have, in the past, said things that are not consistent with this policy" - I place low probability on him significantly changing his mind. You should take this as a sign that you are Straw-manning Eliezer, when you should be Steel-manning him.

Also, how about some creative solutions? An special post tag that must be applied to posts that condone hypoth... (read more)

5kodos9612y

I disagree that this is the entire source of the dispute. I think that even when constrained to optimizing only for good PR, this is an instrumentally ineffective method of achieving that. Censorship is worse for PR than the problem in question, especially given that that problem in question is thus far nonexistent This is trivially easy to do, since the positive effects of enacting the policy are zero, given that the one and only time this has ever been a problem, the problem resolved itself without censorship, via self-policing. Well... the showing him the model part is trivially easy anyway. Convincing him... apparently not so much.

1jbeshir12y

This model trivially shows that censoring espousing violence is a bad idea, if and only if you accept the given premise that censorship of espousing violence is a substantial PR negative. This premise is a large part of what the dispute is about, though. Not everyone is you; a lot of people feel positively about refusing to provide a platform to certain messages. I observe a substantial amount of time expended by organisations on simply signalling opposition to things commonly accepted as negative, and avoiding association with those things. LW barring espousing violence would certainly have a positive effect through this. Negative effects from the policy would be that people who do feel negatively about censorship, even of espousing violence, would view LW less well. The poll in this thread indicates that a majority of people here would be for moderators being able to censor people espousing violence. This suggests that for the majority here it is not bad PR for the reason of censorship alone, since they agree with its imposition. I would expect myself for people outside LW to have an even stronger preference in favour of censorship of advocacy of unthinkable dangerous ideas, suggesting a positive PR effect. Whether people should react to it in this manner is a completely different matter, a question of the just world rather than the real one. And this is before requiring any actual message be censored, and considering the impact of any such censorship, and before considering what the particular concerns of the people who particularly need to be attracted are.

0NancyLebovitz12y

Ambiguity is actually a problem. If people don't know what the policy means, then the person who makes the policy doesn't know what they forbidding or permitting.

2SoftFlare12y

True. I was giving the ambiguity as an example of something people say to claim a policy won't work, without hashing out what that actually means in real execution. Almost every policy is somewhat ambiguous, yet there are many good policies.

0Tenoke12y

There are better ideas in this thread but apparently LW can't afford software changes.

[-]fubarobfusco12y100

Two thoughts:

One: When my partner worked as the system administrator of a small college, her boss (the head of IT, a fatherly older man) came to her with a bit of an ethical situation.

It seems that the Dean of Admissions had asked him about taking down a student's personal web page hosted on the college's web server. Why? The web page contained pictures of the student and her girlfriend engaged in public displays of affection, some not particularly clothed. The Dean of Admissions was concerned that this would give the college a bad reputation.

Naturally the head of IT completely rejected the request out of hand, but was interested in discussing the implications. One that came up was that taking down a student web page about a lesbian relationship would be worse reputation than hosting it could bring. Another was that the IT staff did not feel like being censors over student expression, and certainly did not feel like being so on behalf of the Admissions office.

It's not clear to me that this case is especially analogous. It may be rather irrelevant, all in all.

Two: There is the notion that politics is about violence, not about agreement. That is to say, it is not about what we do whe... (read more)

[-]Incorrect12y100

Would your post on eating babies count, or is it too nonspecific?

http://lesswrong.com/lw/1ww/undiscriminating_skepticism/1scb?context=1

(I completely agree with the policy, I'm just curious)

7quiet12y

We should exempt any imagery fitting of a Slayer album cover, lest we upset the gods of metal with our weakness.

[-]Tenoke12y100

So I finally downvoted Yudkowsky.

5NancyLebovitz12y

What was your line of thought?

[-]Tenoke12y440

That censorship because of what people think of LessWrong is ridiculous. That the negative effect on the reputation is probably significantly less than what is assumed. And that if EY thought that censorship of content for the sake of LW's image is in order he should've logically thought that omitting fetishes from his public OKCupid profile(for the record I've defended the view that this is his right) among other things is also in order as well. And some other thoughts of this kind.

[-]Wei Dai12y90

Eliezer, could you clarify whether this policy applies to discussions like "maybe action X, which some people think constitutes violence, isn't really violence"? And what about nuclear war strategies?

[-]ewbrownv12y80

Censorship is generally not a wise response to a single instance of any problem. Every increment of censorship you impose will wipe out an unexpectedly broad swath of discussion, make it easier to add more censorship later, and make it harder to resist accusations that you implicitly support any post you don't censor.

If you feel you have to Do Something, a more narrowly-tailored rule that still gets the job done would be something like: "Posts that directly advocate violating the laws of in a manner likely to create criminal liability will be deleted... (read more)

[-]twanvl12y80

What if some violence helps reduce further violence? For example corporal punishment could reduce crime (think of Singapore). Note that I am not saying that this is necessarily true, just that we should not a priori ban all discussion on topics like this.

3prase12y

The proposal is to ban such discussions not because violence is bad, but because discussing violence is bad PR. I am pretty sure advocacy of corporal punishment belongs to this category too.

7twanvl12y

Is it really bad PR, though? IMO one of the strengths of LW is that almost any weird topic can be discussed as long as the discussion is rational and civilized. If some interesting posts are banned by the moderators, then this diminishes the value of LW to me.

1prase12y

I don't know whether it's indeed bad PR. It probably depends on one's expectations. I agree with you that banning weird discussions makes LW less attractive (to me, you, certain kind of people), but the site owners want to become more respected by the mainstream and in order to achieve that it is probably a good strategy to remove the weirdest discussions from sight.

1fubarobfusco12y

But the point of LW is not merely having a forum that is valuable to you for discussing weird topics. If you want Reddit, you know where to find it. The point of LW is advancing human rationality, and being a place where people air proposals of violence may get in the way of that. How would we tell? Eliezer and other big names here have been on the receiving end of scandal-sheet gossip-mongering before and may be particularly sensitive to some of these issues. One thing that worries me about this proposal is that Eliezer may be conflating "LW has a bad reputation" with "I have to answer snarky, demeaning questions about foolish things people posted on LW more often than I'm comfortable with" or "People publish articles making fun of my friends and I wish to heck they would stop doing that." But I infer there is also evidence that Eliezer is withholding. But it seems to me that the best way to have a good reputation is to actually be good. For instance, I would like it if people did not see LW as a place to air demeaning, privileged hypotheses (pun intended) about, say, race or gender — in part because many people's evidence standards for these topics is appallingly low; in part because it drives away members of the less-privileged sets (I would rather cooperate with women and defect against PUAs than vice versa; for one thing, there are more women). I would accept the same restriction on discussions of political economy (viz. libertarianism and socialism); although I've talked politics here it's not exactly an area where humans are renowned for being exemplary rationalists.

-1duckduckMOO12y

"is valuable to you for discussing weird topics" "reddit" pick one.

[-]fubarobfusco12y80

Counter-proposal:

We don't contemplate proposals of violence against identifiable people because we're not assholes.

I mean, seriously, what the fuck, people?

7Manfred12y

Generalizations: on average accurate. In specific wrong.

0kodos9612y

Yes, this is the unstated policy we've all been working under up until this point, and it's worked. Which is why it's so irrational to propose a censorship rule.

3fubarobfusco12y

First: "Rational" and "irrational" describe mental processes, not conclusions. A social rule can be useful or useless, beneficial or harmful, well- or ill-defended .... ("If deleting posts that propose violence would benefit Less Wrong, I want to believe that deleting posts that propose violence would benefit Less Wrong. If deleting posts that propose violence would not benefit Less Wrong, I want not to believe that deleting posts that propose violence would ...") Second: Consider the difference between "we're not assholes" and "we don't want to look like assholes". Or between "I will cooperate" and "I want you to think that I will cooperate." A defector can rationally conclude the latter, but not the former (since it is false of defectors).

[-]Mestroyer12y80

I'll restate a third option here that I made in the censored thread (woohoo, I have read a thread Eliezer Yudkowsky doesn't want people to read, and that you, dear reader of this comment, probably can't!) Make an option while creating a post to have it be only viewable by people with certain karma or above, or so that after a week or so, it disappears from people without that karma. This is based on an idea 4chan uses, where it deletes all threads after they become inactive, to encourage people to discuss freely.

This would keep these threads from showing up when people Googled LessWrong. It could also let us discuss phyggishness without making LessWrong look bad on Google.

[-]drethelin12y190

Yes, and if we all put on black robes and masks to hide our identities when we talk about sinister secrets, no one will be suspicious of us at all!

[-]NancyLebovitz12y140

You can't reliably make things on the internet go away.

3Mestroyer12y

You can make them hard enough to access that they won't be stumbled upon by random people wondering what LessWrong is about, which is basically good enough for preserving LessWrong's reputation.

3NancyLebovitz12y

I was thinking about people posting screen shots.

6Qiaochu_Yuan12y

Agreed. It only takes one high-karma user posting a screenshot on reddit of LW's Secret Thread Where They Discuss Terrorism or whatever...

-2kodos9612y

I can think of a few different ways, requiring no more than a few dozen software-engineer-hours, that this could be solved effectively enough to make it a non-issue.

[-]fubarobfusco12y190

If my browser displays it as text, I can copy it. If you try dickish JavaScript hacks to stop me from copying it the normal way, I can screenshot it. If you display it as some kind of hardware-accelerated DRM'd video that can't be screenshotted, I can get out a fucking camera and take a fucking picture. If I post it somewhere and you try to shut me down, you invoke the Streisand Effect and now all of Reddit wants (and has) a copy, to show their Censorship Fighter status.

tl;dr: No, you can't stop people from copying things on the Internet.

3kodos9612y

Of course. But a "good enough" solution to the stated problem doesn't need to be able to do that. There are a number of different approaches I can think of off the top of my head, in increasing order of complexity: * Just keep it from getting indexed by google, and expire it after a certain period. Sure, a sufficiently determined attacker could just spider LW every day, but do we actually think there's an organized conspiracy out there against us? * Limit access to people who can be trusted not to copy it - either based on karma as suggested, or individual vetting. I'm not a fan of this option, but it could certainly be made to work, for certain values of "work". * Implement a full on OTR style system providing full deniability through crypto. Rather than stopping content from being copied, just make sure you can claim any copy is a forgery, and nobody can prove you wrong. A MAJOR engineering effort of course, but totally possible, and 100% effective.

3handoflixue12y

I can't help but see two major flaws: 1) If I link to a major, encrypted offshoot of LessWrong, people will AUTOMATICALLY be suspicious and it will damage PR. 2) Why would it be any easier to cry "it's a forgery" in this situation versus me posting a screenshot of an unencrypted forum? o.o Especially given #1... 3) I can share my password / decryption key / etc..

-2kodos9612y

Well, point 3 can be eliminated by proper use of crypto. See OTR The response to point 2 is that by having it be publicly known to everyone that messages' contents are formally mathematically provably deniable (as can be guaranteed by proper crypto implementation), that disincentivizes people from even bothering to re-post content in the first place. Point 1, however, I agree with completely, and that's why I'm not actually advocating this solution.

0handoflixue12y

You're flat-out wrong about #3. Encryption is just a mathematical algorithm, it doesn't care who uses it, only that you have the key. In short, encryption is just a very complex function, so you feed Key + Message in, and you get an Output. f(K,M) = O I already have access to Key and Message, so I can share both of those. The only thing you can possibly secure is f(). If you have a cryptographic program, like OTR, I can just decompile it and get f(), and then post a modified version that lets the user manually configure their key (I think this is actually trivial in OTR, but it's been years since I poked at it) If it's a website where I login and it auto-decrypts things for me, then I can just send someone the URL and the key I use. Point 2 seems to reply on Point 3, and as far as I'm aware the only formally mathematically provably deniable method WHEN THE KEY IS COMPROMISED is a one-time pad. I'm not sure how much crypto experience you have, but "and no one else knows the key" is a foundation of every algorithm I have ever worked on, and I'm reasonably confident that it's a mathematical requirement. I simply cannot imagine how you could possibly write a crypto algorithm that is secure EVEN with a compromised key. EDIT: If you still think I'm wrong, can you please give me a sense of your crypto experience? For reference: I've met with the people who wrote OTR and hang out in a number of crypto circles, but only do fairly basic stuff in my actual work. I do still have a hobby interest in it, and follow it, but the last time I did any serious code breaking was about a decade ago.

-2kodos9612y

You seem to be using a very narrow definition of "crypto".. I'm not sure whether you're just being pedantic about definitions, in which case you may be correct, or if you're actually disputing the substance of what I'm saying. To answer your question, I'm not a cryptographer, but I have a CS degree and am quite capable of reading and understanding crypto papers (though not of retaining the knowledge for long)... it's been several years since I read the relevant papers, so I might be getting some of the details wrong in how I'm explaining it, but the basic concept of deniable message authentication is something that's well understood by mainstream cryptographers. You seem to be aware of the existence of OTR, so I'm confused - are you claiming that it doesn't accomplish what it says it does? Or just that something about the way I'm proposing to apply similar technology to this use case would break some of its assumptions? The latter case is entirely possible, as so far I've put a grand total of about 5 minutes thought into it... if that's the case I'd be curious to know what are the relevant assumptions my proposed use case would break?

0handoflixue12y

If I give you my key, you can pretend to be me on OTR. I've had friends demonstrate this to me, but I've never done it myself, so 99% confidence. Technical disagreement, as near as I can tell, since you're not advocating for the solution.

1roystgnr12y

This must be why the media companies haven't given up on DRM yet. They think if they can just unmask and arrest the ringleaders of the "organized conspiracy out there" then copy protection will start working, when in reality any random person can become a "conspiracy" member with nothing more than a little technical knowledge, a little free time, and a moral code that encourages copying. To be fair, the "vetting" and "full deniability" options don't really apply to the ??AA. The best pre-existing example for those kinds of policies might be the Freemasons or the Mormons? In neither case would I be confident that the bad PR they've avoided by hiding embarrassing things hasn't been worse than the bad PR they've abetted by obviously dissembling and/or by increasing the suspicion that they're hiding even worse things.

0kodos9612y

Exactly. That's why I'm not actually advocating any of these technical solutions, just pointing out that they do exist in solution-space. The solution that I'm actually advocating is even simpler still: do nothing. Rely on self-policing and the "don't be an asshole" principle, and in the event that that fails (which it hasn't yet), then counter bad speech with more speech: clearly state "LW/SIAI does not endorse this suggestion, and renounces the use of violence." If people out there still insist on slandering SIAI by association to something some random guy on LW said, then fuck em - haters gonna hate.

4Tenoke12y

Not a bad option indeed. It has a merit if we are really that bothered about the general view of LW. And for the record the post is still accessible albeit deleted.

2Eliezer Yudkowsky12y

LW has effectively zero resources to implement software changes.

3kodos9612y

If this were your real rejection, you would be asking for volunteer software-engineer-hours.

7Eliezer Yudkowsky12y

Tried.

[-]gelisam12y150

Are you kidding? Sign me up as a volunteer polyglot programmer, then!

Although, my own eagerness to help makes me think that the problem might not be that you tried to ask for volunteers and didn't get any, but rather that you tried to work with volunteers and something else didn't work out.

[-]yli12y130

Maybe it's just that volunteers that will actually do any work are hard to find. Related.

Personally, I was excited about doing some LW development a couple of years ago and emailed one of the people coordinating volunteers about it. I got some instructions back but procrastinated forever on it and never ended up doing any programming at all.

4gelisam12y

I understand how that might have happened. Now that I am no longer a heroic volunteer saving my beloved website maiden, but just a potential contributor to an open source project, my motivation has dropped. It is a strange inversion of effect. The issue list and instructions both make it easier for me to contribute, but since they reveal that the project is well organized, they also demotivate me because a well-organized project makes me feel like it doesn't need my help. This probably reveals more about my own psychology than about effective volunteer recruitment strategies, though.

7Risto_Saarelma12y

The site is open source, you should be able to just write a patch and submit it.

-1kodos9612y

This would be a poor investment of time without first getting a commitment from Eliezer that he will accept said patch.

4Risto_Saarelma12y

It'd get you familiar with the code base, which you'd need to be anyway if you wanted to be a volunteer contributor.

3gelisam12y

After finding the source and the issue list, I found instructions which indicate that there is, after all, non-zero engineering resources for lesswrong development. Specifically, somebody is sorting the incoming issues into "issues for which contributions are welcome" versus "issues which we want to fix ourselves". The path to becoming a volunteer contributor is now very clear.

4AdeleneDawner12y

Getting someone to sort a list, even on an ongoing basis, is not functionally useful if there's nobody to take action on the sorted list.

0handoflixue12y

I like the idea, but I have to agree that the PR cost of such a thing being leaked is probably vastly worse than simply being open about it in the first place.

[-]jimrandomh12y80

Posts advocating or "asking about" violence against identifiable real people or groups should be deleted at the admins' discretion:

[pollid:374]

Posts advocating or "asking about" violation of laws that are actually enforced against middle-class people, other than the above, should be deleted at the admins' discretion:

[pollid:375]

[-]gjm12y100

This poll, like EY's original question, conflates two things that don't obviously belong together. (1) Advocating certain kinds of act. (2) "Asking about" the same kind of act.

I appreciate that in some cases "asking about" might just be lightly-disguised advocacy, or apparent advocacy might just be a particularly vivid way of asking a question. I'm guessing that the quotes around "asking about" are intended to indicate something like the first of these. But what, exactly?

3jbeshir12y

I think in this context, "asking about" might include raising for neutral discussion without drawing moral judgements. The connection I see between them is that if someone starts neutral discussion about a possible action, actions which would reasonably be classified as advocacy have to be permitted if the discussion is going to progress smoothly. We can't discuss whether some action is good or bad without letting people put forward arguments that it is good.

3gjm12y

There's certainly a connection. I'm not convinced the connection is so intimate that if censoring one is a good idea then so is censoring the other.

3jimrandomh12y

...but it'd be nice to have a poll to point at later, to show consensus, and I'd be surprised if people disagreed.

-7Eliezer Yudkowsky12y

[-]handoflixue12y70

It seems like this represents, not simply a new rule, but a change in the FOCUS of the community. Specifically, it used to be entirely about generating good ideas, and you are now adding a NEW priority which is "generating acceptable PR".

Quite possibly there is an illusion of transparency here, because there hasn't really BEEN (to my knowledge) any discussion about this change in purpose and priorities. It seems reasonable to be worried that this new priority will get in the way of, or even supersede the old priority, especially given the phrasin... (read more)

7Qiaochu_Yuan12y

When you go on a first date with someone, would you tell them "hey, I've got this great idea about how I should [insert violence here] in order to [insert goal here]. What do you think?" Of course not, because whether or not this is a good idea, you are not getting a second date. PR isn't inherently Dark Arts. It's about providing evidence to another party about yourself or your organization in a way which is conducive to further provision of evidence. If you start all your dates by talking about your worst traits first, you aren't giving your date incentives to stick around and learn about your best traits. If LW becomes known for harboring discussions of terrorism or whatever, you aren't giving outsiders incentives to stick around and learn about all the other interesting things happening on LW, or work for SIAI, etc.

2handoflixue12y

You'd be amazed how many second dates I get... That said, I don't think PR is Dark Arts, I just think it's an UNSPOKEN change in community norms, and... from a PR standpoint, this post is a blatantly stupid way of revealing that change. Huh. Either the original post is bad because PR is bad, or this post is bad because it demonstrates bad PR. Lose/lose :)

1DanArmak12y

This begs the question by assuming the proposed violence is a bad trait.

3Qiaochu_Yuan12y

All I'm assuming is that a typical date will assume that people who talk about violence on the first date are crazy and/or violent themselves. This is an argument about first impressions, not an argument about goodness or badness.

0ChristianKl12y

If I would go to a date with a girl who believes in the necessity of a communist revolution I wouldn't judge her negatively for that political belief. There are character traits that I would judge much worse.

1Qiaochu_Yuan12y

Okay, but 1) the fact that you post on LW is already evidence that you're not representative of the general population in various ways, and 2) communist revolution is at least an idea that people learn about in college, and it's not too unusual to hear a certain type of person say stuff like that. I had in mind the subject of the deleted post; if a typical person heard someone talking like that, their first reaction would be that that person is crazy, and with a reasonable choice of priors this would be a reasonable inference to make.

5ChristianKl12y

I haven't read the deleted post. If someone who knew what the case is about would write it to me via private message I would appreciate it. The communist revolution is a classic example of an idea that involves the advocation of illegal violence against a specific group of people. There are certainly internet forums where that kind of political speech isn't welcome and will get deleted. On LessWrong I think that's a position that should be allowed to be argued. Moldbuggian advocacy of a coup d'état should also be allowed. Some people might think that you are crazy if you argue Moldbuggianism on a first date. At the same time I think that idea should be within the realm of permissable discourse on LessWrong.

0DanArmak12y

If LW-compatible people are more welcoming of discussion of violence than the general population, then the bad PR would affect them less than it would other people, so we should care less about bad PR.

[-]Eliezer Yudkowsky12y50

If there's a general policy against discussing violence on LW, and I can point to statements from the same timeframe of mine condemning such violence, it may help. It may not. Reporters are stupid. Your argument does not actually say why the anti-violence-discussion policy is a bad idea, and seems to be ad hominem tu quoque.

[-]Eliezer Yudkowsky12y50

Everyone even slightly famous gets arbitrary green ink. Choosing which green ink to 'complain' about on your blog, when it makes an idea look bad which you would find politically disadvantageous, is not a neutral act. I'm also frankly suspicious of what the green ink actually said, and whether it was, perhaps, another person who doesn't like the "UFAI is possible" thesis saying that "Surely it would imply..." without anyone ever actually advocating it. Why would somebody who actually advocated that, contact Ben Goertzel when he is kn... (read more)

9saturn12y

If that's the case, it seems like giving him the title Director of Research could cause a lot of confusion. I certainly find it confusing. Maybe that was a different Ben Goertzel?

2timtyler12y

Reportedly, Ben Goertzel and OpenCog were intended to add credibility through association with an academic:

2Eliezer Yudkowsky12y

Honestly, at this point I'm willing to just call that a mistake on Tyler Emerson's part.

[-]Decius12y50

Why the explicit class distinction?

It would be prohibited to discuss how to or speed and avoid being cited for it. (I thought that this was already policy, and I believe it to be a good policy.)

It would not be prohibited to discuss how to be a vagrant and avoid being cited for it. (Middle class people temporarily without residences typically aren't treated as poorly as the underclass.)

Should the proper distinction be 'serious' crimes, or perhaps 'crimes of infamy'?

0DanArmak12y

As judged by who? (I don't endorse EY's original proposal, either.)

0Decius12y

As judged by the person making the decision to delete.

1DanArmak12y

I don't think the words "serious crime" have the property that different judges would make very similar judgements about a given discussion.

0Decius12y

Is that phrase better or worse than

1DanArmak12y

"Laws that are actually enforced" is at least an empirical question. "Serious crime" is just a value judgement.

0Decius12y

"Middle class" is just as much an undefined term as "serious crime". It's concerning that we are having trouble agreeing on where the edge cases are, much less how to decide them.

[-]blacktrance12y40

I don't have any principled objection to this policy, other than that as rationalists, we want to have fun, and this policy makes LW less fun.

[-]Larks12y40

Does advocating gun control, or increased taxes, count? They would count as violence is private actors did them, and talking about them makes them more likely (by states). Is the public-private distinction the important thing - would advocating/talking about state-sanctioned genocide be ok?

8ikrase12y

While an interesting question, I think that the answer to that is reasonably obvious.

3Eugine_Nier12y

What about capital punishment and/or corporal punishment?

0Luke_A_Somers12y

To call either gun control or taxation violence is stretching matters beyond reasonable limits. The only sense in which they are is the sense in which any public policy is - that it is backed by the government. If anything to do with the government has to be considered as 'about violence'... bah.

0DanArmak12y

Of course all laws enforced by governments are enforced with the threat of violence, and actual violence against violators. The law itself may not be violent, but violence will be used if necessary to enforce it. Violence is not necessarily bad; it is a tool that may be the right one to use. Just as government and laws are not bad in themselves. If you object to saying government uses violence, you must be disagreeing with me over the meaning of the word "violence".

0Luke_A_Somers12y

Sufficiently proximate to the point that simply talking about government is in fact a derail advocating violence? I think not.

0DanArmak12y

Of course simply talking about government, or any particular government policy, is not about violence. And so it's not a derail that needs to be moderated. But violence is an essential part of government. That's all I was saying. Compare: simply talking about cryonics is not about quantum mechanics. If discussion of quantum mechanics were counterfactually forbidden, talking about cryonics would not be forbidden thereby. But cryonics, like all physical systems, is "implemented" or backed by quantum mechanics; you can't have one without the other.

-4kodos9612y

I don't think it's silly, and based on the LW survey results, neither do approximately 30.3% of LW users. But aside from that, OP said "More generally: Posts or comments advocating or 'asking about' violation of laws that are actually enforced against middle-class people". Gun control (though not taxation) clearly falls under this illegality clause, without resort to classifying it as "violence".

6jsalvatier12y

I identify as libertarian and have been objectivist, but calling taxation theft (and other similar claims) is almost always sneaking in connotations.

0kodos9612y

True, but that doesn't change the fact that the wording of the proposed policy is heavily subject to interpretation, which is the point I was trying to make.

6Luke_A_Somers12y

'Libertarian' does not mean 'believes all government action is violence'.

0kodos9612y

In the event of gun control, it would in fact be illegal even if done by a state actor. Edit: assuming USA of course.

[-]drethelin12y30

Regardless of whether you think censoring is net good or bad (for the forums, for the world, for SIAI), you have to realize the risks are far more on Eliezer than they are on any poster. His low tolerance responses and angry swearing are exactly what you should expect of someone who feels the need to project an image of complete disassociation from any lines of speculation that could possibly get him in legal trouble. Eliezer's PR concerns are not just of the forums in general. If he's being consistent, they should be informing every one of his comments on... (read more)

4ChristianKl12y

PR is something different than legal trouble. Eliezer didn't express any concern that the speech he wants to censor would create legal trouble. If Eliezer wants to make that argument I should make that argument directly instead of speaking about PR. In the US speech that encourages violent conduct is protect under the first amendment. See Brandenburg v. Ohio. If you actually to want to protect against legal threats you should also forbid a lot of discussion that could fall under UK libel law.

2kodos9612y

This, I think, is the fundamental point of diagreement here. The emotional valence is far greater on Eliezer than on us, but if we're taking seriously the proposition that the singularity is coming in our lifetimes (and I do), then the risks are the same for all of us. Angry swearing? Did I miss some posts? Link please. I suppose I should point out that when I referred earlier to Eliezer's occasional lapses in judgement, I was absolutely NOT intending to refer to this. I wasn't actively commenting at the time, but looking back on that episode, I found a lot of the criticism of Eliezer regarding his OKcupid profile to be downright offensive. When I first read the profile, I was actually incredibly impressed by the courage he displayed in not hiding anything about who he is.

0TheOtherDave12y

In my experience, it's difficult to display a high level of courage about revealing the truth about myself and at the same time commit to moderating the image I present so as to avoid public-relations failures. At some point, tradeoffs become necessary.

-1drethelin12y

http://lesswrong.com/lw/g24/new_censorship_against_hypothetical_violence/84fd but I think there's others I think it's fairly reasonable for Eliezer to guard against large risks for himself in exchange for small and entirely theoretical risks to the effectiveness of the forums. I don't think this censorship decision has a very meaningful impact either way on FAI. You can also factor his own (possibly inflated) evaluation of his value to the cause. A 0.01 percent chance of Eliezer getting jailed might be worse than 10 percent chance of stifling a mildly useful conversation or having someone quit the forums due to censorship.

-4kodos9612y

I disagree. One of the most common objections to the idea of FAI/CEV is "so will this new god-like AI restrict fundamental civil liberties after it takes over?"

5drethelin12y

I've never heard this objection. Fundamental civil liberties is also a fundamentally diseased concept.

0ChristianKl12y

If you explain that position in huge detail, there a plausible chance that it includes advocation of illegal conduct and could therefore be censored through this policy.

1Desrtopa12y

Keep in mind that the policy is going to be done through human implementation with the specific intention of avoiding inconveniently broad interpretations like this. It's not enough to show that the censorship policy could theoretically be used to stifle conversation we actually want here, the important question is whether it actually would be.

0ChristianKl12y

I think that's a very dangerous idea. This community is about developing FAI. FAI should be expected to act according to the rules that you give it. I think the policy should be judged by the way it would actually work if it would be applied as it's proposed. There also the problem of encouraging group think: Advocating of illegal conduct gets censored when it goes against the morality of this group but is allowed when it's within the realms of that morality is bad. This community should have consistent rules about which discussions are allowed and which aren't. Censoring on a case by case is problematic. If you start censoring certain speech that advocates violence and avoid censoring other speech that advocates violence you also have the problem that you get more responsibility for the speech that you allow. In the absence of a censorship policy you don't endorse a viewpoint by not censoring the viewpoint. If you however do censor, and you do make decisions not to censor specific speech that's an endorsement of that speech.

0Desrtopa12y

The way it's proposed is to be applied according to the judgment of a moderator. It makes no sense to pretend that we're beholden to the strictest letter of the rule when that's not how it's actually going to work. What speech that advocates violence do you think would get a pass while the rest would get censored?

1ChristianKl12y

I don't know excatly how much speech Eliezier wants to censor. I wrote a post with a bunch of examples. I would like to see with of those example Eliezer considers worthy of censorship.

0kodos9612y

Please explain. (I've heard this argued before, but I'm curious what your particular angle on it is)

5Jayson_Virissimo12y

He is probably pattern-matching "fundamental civil liberties" to Natural Rights, which are not taken very seriously around these parts, since they are mostly myth.

[-]buybuydandavis12y30

If the point was to "make a good impression" by distorting the impression given by people on the list to potential donors, maybe a more effective strategy is to shut up and do it, instead of making an announcement about it and causing a ruckus. "Disappear" the problems quietly and discretely.

This reminds me of the phyg business. Prompting long discussion threads about how "We are not a phyg! We are not a phyg!" is not recommended behavior if you don't want people to think you're not a phyg.

0Qiaochu_Yuan12y

People notice when this happens, and the resulting uproar might have been worse; then accusations would be flying about lack of transparency.

1buybuydandavis12y

Might have been. I didn't see it, and I didn't see any brouhaha over a deleted post. Was there one? More than the 318 comments in this thread announcing the policy? I see lots of downvotes for EY. I don't think it's going well. The advantage of just doing it is that many people will not notice it at all, and those that notice it have seen the offending post, and so have a concrete context to go by. When people haven't seen the threat, they talk about censorship. My guess is that in a concrete case, at worst it will come across as an overreaction to some boorish behavior. If the concrete case really involved stifling ideas, I'd expect people to make a huge stink about it. The big stink about poilcy, and undetected by my stink about the concrete case tells me that people are getting their undies in a bunch over nothing.

[-]NancyLebovitz12y30

Definition of green ink

[-]Eliezer Yudkowsky12y30

If combined with a "Please write him and ask him to shut down!", sure. I think it's understood by default in most civilized cultures that violence is not being advocated by default when other courses of action are being presented. If the action to be taken is mysteriously left unspecified, it'd be a judgment call depending on other language used.

[-]Qiaochu_Yuan12y30

I wouldn't have posted the following except that I share Esar's concerns about representativeness:

I think this is a good idea. I think using the word "censorship" primes a large segment of the LW population in an unproductive direction. I think various people are interpreting "may be deleted" to mean "must be deleted." I think various people are blithely ignoring this part of the OP (emphasis added):

In other words, the form of this discussion is not 'Do you like this?' - you probably have a different cost function from peop

... (read more)

-2kodos9612y

I'm not underestimating that at all... I'm saying that this policy makes us look bad... WAY worse than the disease it's intended to cure, especially in light of the fact that that disease cleared itself up in a few hours with no intervention necessary.

[-]kodos9612y30

How about instead of outright censorship, such discussions be required to be encrypted, via double-rot13?

-2[anonymous]12y

Rot13 applied twice is just the original text...

[-]kodos9612y100

..............whooosh................

-1[anonymous]12y

In light of the above getting upvotes, I'm not sure if it's the "whoosh" of double-rot13 going over your head as I originally thought, or if it's indicating intended sarcasm going over my head, or some other meaning not readily obvious to me (inferential distance and all that.)

3Manfred12y

Yes, the original comment was a joke.

[-]wedrifid12y30

Yes, a post of this type was just recently made. I will not link to it, since this censorship policy implies that it will shortly be deleted, and reproducing the info necessary to say who was hypothetically targeted and why would be against the policy.

Someone please send me a link via PM? Or perhaps the author could PM me? Not because the censorship of that class bothers me but because talking to wedrifid is not posting things on the internet, I'm curious and there are negligible consequences for talking to me about interesting hypothetical questions.

(Disregard the above is the post or comment was boring.)

[-][anonymous]12y300

tl;dr: tobacco kills more people than guns and cars combined. Should we ?

PS: fuck the police

9wedrifid12y

(I laughed). Thanks nyan. (I hope this kind of satirical summary is considered acceptable.)

9kodos9612y

This kind of uncertainty about what is and is not acceptible, is perhaps the primary reason why such censorship policies are evil.

5Viliam_Bur12y

This is a huge exaggeration! I mean, yes, in a far mode, censorship creates fear and so on... but let's come back to near mode and ask: "What is the worst consequence of stepping just a little on the wrong side of this uncertain line?" Well, Eliezer would delete my comment or article, and that's it. It does not really make my legs shake. My guess is that "tobacco kills more people than guns and cars combined. Should we ?", written literally like this, is acceptable. Probability estimate? It would be 98% in a different discussion on a different day, and perhaps 95% here and now because Eliezer may still be in the deleting mood. So what? If I am wrong, he will delete that comment, and perhaps also my comment for quoting it. And that's all. Am I afraid? No. Actually, I would probably not even notice if that happened. Generally, I also prefer precise rules to imprecise ones, but there are limits how precise one can be in topics like this. Trying to make the rules exact (to avoid all harm, but allow all harmless discussion) is a FAI-complete problem. Even the real-world laws often have imprecise parts. Also, the more precise rules, the greater pressure on moderators to follow them literally; but I would prefer them using their own judgement.

0ChristianKl12y

Eliezer should engage into any deliberate practice that involves solving FAI-complete problems and not shun away from putting thinking into them and leave vague rules because the problem is too hard.

1William_Quixote12y

When time passes and the above post is not censored uncertainty will decrease. Arguments about chilling effects due to uncertainty are probably systemically overstated because the about of uncertainty we have now is much higher than the average uncertainty over the life of the policy. Also the post in question is great. P.S. fuck the police.

8CronoDAS12y

As the author of the offending Discussion post in question, I'd say it's an adequate summary.

[-]Shmi12y30

Would it censor a discussion of, say, compelling an AI researcher by all means necessary to withhold their research from, say, the military?

[-]Eliezer Yudkowsky12y130

Yes. This seems like yet another example of "First of all, it's a bad fucking idea, second of all, talking about it makes everyone else look bad, and third of all, if hypothetically it was actually a good idea you'd still be a fucking juvenile idiot for blathering about it on the public Internet." What part of "You fail conspiracies forever" is so hard for people to understand? Talk like this serves no purpose except to serve as fodder for people who claim that leads to violence and is therefore false, and your comment shall be duly deleted once this policy is put into place.

-2kodos9612y

I don't see how this comment even fits the proposed policy, except under a motivatedly-broad reading of "by all means necessary"

6CarlShulman12y

Wikipedia thinks otherwise:

-7kodos9612y

4Paul Crowley12y

Just checked with my houseguest; his interpretation is also "a call to violence".

[-]NancyLebovitz12y20

The overall emotional tone. The lack of calls to direct action. The encouragement to think about the effects of one's actions, with thinking including that you take an honest look at opposing points of view.

[-]Kevin12y20

What would the response to this have been if instead of "censorship policy" the phrase would have been "community standard"?

3katydee12y

It probably would have been more positive but less honest.

[-]DanArmak12y10

laws that are actually enforced against middle-class people

Different countries can have very different laws. Are you going to enforce this policy with reference to U.S. laws only, as they exist in 2012? If not, what is your standard of reference?

As I commented elsewhere, if your goal is to prevent bad PR, it is not obvious to me that this policy is the right way to optimize for it. Perhaps you have thought this out and have good reasons for believing that this policy is best for this goal, but it is not clear to me, so please elaborate on this if you can.

[-]MugaSofer12y10

So ... we can't discus assassination the president, that seems fine if unnecessary. But we can't debate kidnapping people and giving the ransom to charity without the "pro-" side being deleted? That seems like it will either destroy a lot of valuable conversations for minimal PR gain, or be enforced inconsistently, leading to accusations of bias. Probably both.

[-]handoflixue12y10

"anyone talking about a proposed crime on the Internet fails forever as a criminal"

I realize this isn't your TRUE objection, just a bit of a tangential "Public Service Announcement". The real concern is simply PR / our appearance to outsiders, right? But... I'm confused why you feel the need to include such a PSA.

Do we have a serious problem with people saying "Meet under the Lincoln Memorial at midnight, the pass-phrase is Sic semper tyrannis" or "I'm planning to kill my neighbor's dog, can you please debug my plot, I... (read more)

-1[anonymous]12y

Look. He's a man who's earning the money by fear mongering. Passably bright which makes it so much worse from moral standpoint. He has opportunity to actually condemn violence. But that would require him to apologize for himself posting very inflammatory statements about other people's projects, because by the moral standards, those are equally bad. So he chooses to make it abundantly clear it's only for PR and no not really. (as on the point, there was a post by someone which can be summarized as "tobacco kills a lot of people, utilitarian bla bla, why not off tobacco executives")

[-]NancyLebovitz12y10

LessWrong attracts all sorts of people including would-be unabombers of varying level of dedication.

Why do you think LW attracts would-be unabombers?

[-]Dahlen12y10

(I seriously should've posted this question back when the thread only had 3 comments.)

I have no qualms about the policy itself, it's only commonsensical to me; my question is only tangentially related:

Do you believe "censorship" to be a connotatively better term than "moderation"?

3fubarobfusco12y

To me this sounds like a mix of counter-signaling and a way of saying, "Yes, this proposal is controversial, as policy debates should not appear one-sided. There are ups and downs. There are reasons to believe that some of the negatives that obtain to other things called 'censorship' may happen here."

[-]buybuydandavis12y00

"Reason: Talking about such violence makes that violence more probable,"

Does it? In some cases yes, and some cases no. Wacky people advocating violence can be smacked down by the crowd. Wacky violent loners need perspective from other people.

Talking about socially approved violence probably makes it more likely. Talking about socially disapproved violence might make it less likely.

The problem with these conversations are the generalities involved. We don't have examples of the offending material, and worse, the whole point is to hide the exampl... (read more)

[-]handoflixue12y00

I'd ask if there's any evidence of removal but... I can't imagine anything other than Eliezer saying "Yep, I deleted it" would do the trick.

Well, actually, does this software ALLOW deletions, or does deleted comment get replaced with a "Deleted Content" placeholder? Because if it's the latter, a link to that placeholder would be decent evidence :)

0Eugine_Nier12y

It only gets replaced with a place holder if there are replies.

0Document12y

Do deleted comments still appear on the list of comments by that user?

0Eugine_Nier12y

I'm not sure. I suspect it might depend on who deleted them.

[-]JonathanLivengood12y-10

Do you have worked out numbers (in terms of community participation, support dollars, increased real-world violence, etc.) comparing the effect of having the censorship policy and the effect of allowing discussions that would be censored by the proposed policy? The request for "Consequences we haven't considered" is hard to meet until we know with sufficient detail what exactly you have considered.

My gut thinks it is unlikely that having a censorship policy has a smaller negative public relations effect than having occasional discussions that vio... (read more)

[-]Epiphany12y-40

If virtualizing people is violence (since it does imply copying their brains and, uh, removing the physical original) you may want to censor Wei_Dai over here, as he seems to be advocating that the FAI could hypothetically (and euphemistically) kill the entire population of earth:

Wei Dai's Ironic Security Idea

5Wei Dai12y

My hypothetical scenario was that replacing a physical person with a software copy is a harmless operation and the FAI correctly comes to this conclusion. It doesn't constitute hypothetically (or euphemistically) killing, since in the scenario, "virtualizing" doesn't constitute "killing".

-4Epiphany12y

That is your exact wording. Not "In the event that the AGI determines that it's safe to [euphemism for doing something that could mean killing the entire human race] because there are software copies." or "if virtualizing is safe..." Even if your wording was that, I'd still disagree with it. I thought the most important reason to do friendliness research was to give the AGI what it needs to avoid making decisions that could kill all of humanity. It is humanity's responsibility to dictate what should happen in this case and ensure that the AGI understands enough to choose the option we dictate. If you aren't in favor of micromanaging the millions of tiny ethical decisions it will have to make like exactly how many months to put a lawbreaker in jail, that's one thing. If you aren't in favor of making sure it decides correctly on issues that could kill all of humanity, that's negligent beyond imagining. If you are aware of a decision that an AGI could make that could kill all of humanity, and you are in favor of creating an AGI that hasn't been given guidance on that issue, then you're in favor of creating a very dangerous AGI. Advocating for an AGI that will kill all of humanity vs. advocating for an AGI that could kill all of humanity is a variation on "advocating violence" (it's advocating possible violence) but, to me, it's no different from saying: "I'm going to put one bullet in my gun, aim at so-and-so, and pull the trigger!" - Just because the likelihood of killing so-and-so is reduced to 1 in 6 from what's more or less a certainty does not mean it's not a murder threat. Likewise, adding the word "possibly" into a sentence that would otherwise break the censorship policy is a cheap way of trying to get through the filter. That should not work. "We should possibly go on a killing rampage." - no. What's most alarming is that you've done work for SIAI. The whole point of SIAI is not to go "Let's let the AGI decide what is ethical" but "Let's iron out all the

2Wei Dai12y

As I clarified in a subsequent comment in that thread, "if the FAI concludes that replacing a physical person with a software copy isn't a harmless operation, it could instead keep physical humans around and place them into virtual environments Matrix-style." We could argue about whether to build an FAI that can make this kind of decision on its own, but I had no intention of doing anyone any harm. Yes the attempted-FAI may reach this conclusion erroneously and end up killing everyone, but then any method of building an FAI has the possibility of something going wrong and everyone ending up dead. I've never received any money from them and am not even a Research Associate. I have independently done work that may be useful for SIAI, but I don't think that's the same thing from a PR perspective. Actually I think SIAI's official position is something like "Let's work out all the problems involved in letting the AGI decide what is ethical." If you disagree with this, let's argue about it, but could you please stop saying that I advocate killing people?

-2Epiphany12y

I know. reviews my wording very carefully "If virtualizing people is violence ... Wei_Dai ... seems to be advocating " "Advocating for an AGI that will kill all of humanity (in the context of this is not what you said) vs. advocating for an AGI that could kill all of humanity (context: this is what you said)" My understanding is that it's your perspective that copying people and removing the physical original might not be killing them, so my statements reflect that but maybe it would make you feel better if I did this: "If virtualizing people is violence ... Wei_Dai ... seems to be advocating ... kill the entire population of earth (though he isn't convinced that they would die)" And likewise with the other statement. Sorry for the upset that has probably caused. It wasn't my intent to accuse you of actually wanting to kill everyone. I just disagree with you and am very concerned about how your statement looks to others with my perspective. More importantly, I feel concerned about the existential risk if people such as yourself (who are prominent here and connected with SIAI) are willing to have an AGI that could (in my view) potentially kill the entire human race. My feeling is not that you are violent or intend any harm, but that you appear to be confused in a way that I deem dangerous. Someone I'm close to holds a view similar to yours and although I find this disturbing, I accept him anyway. My disagreement with you is not personal, it's not a judgment about your moral character, it's an intellectual disagreement with your viewpoint. I think the purpose of this part is to support your statement that you have no intention to harm anyone, but if it's an argument against some specific part of my comment, would you mind matching them up because I don't see how this refutes any of my points. It's not easy for me to determine your level of involvement from the website. This here suggests that you've done important work for SIAI: http://singularity.org/blog/2

1Wei Dai12y

I think it may not be possible for humans to determine this, in the time available before someone builds a UFAI or some other existential risk occurs. Still, I have been trying to determine this, for example just recently in Beware Selective Nihilism. Did you see that post? Were you serious about having Eliezer censor my comment? If so, now that you have a better understanding of my ideas and relationship with SIAI, would you perhaps settle for me editing that comment with some additional clarifications?

-2Epiphany12y

Sorry for not responding sooner. The tab explosion triggered by the links in your article and related items was pretty big. I was trying to figure out how to deal with the large amount of information that was provided. If you want to consider my take on it uniformed, fine. I haven't read all of the relevant information in the tab explosion (this would take a long time). Here is my take and my opinion on the situation: If a person is copied, the physical original will not experience what the copy experiences. Therefore, if you remove the physical original, the physical original's experiences will end. This isn't perfectly comparable to death seeing as how the person's experiences, personality, knowledge, and interaction with the world will continue. However, the physical original's experiences will end. That, for many, would be an unacceptable result of being virtualized. I believe in the right to die, so regardless of whether I think being virtualized should be called "death", I believe that people have the right to choose to do it to themselves. I do not believe that an AGI has the right to make that decision for them. To decide to end someone else's experiences without first gaining their consent qualifies as violence to me and it is alarming to see someone as prominent as you advocating this. My opinion is that it's better for PR for you to edit your comment. Even if, for some reason, reading the entire tab explosion would somehow reveal to me that yes, the physical original would experience what the copy experiences even after being destroyed, I think it is likely that people who have not read all of that information will interpret it the way that I did and may become alarmed especially after realizing that it was you who wrote this. I would be really happy to see you edit your own "virtualize everyone" comments. I do think something needs to be done. My suggestion would be to either: A. Clearly state that you believe the physical original will experience

1Wei Dai12y

I added a disclaimer to my comment. I had to write my own since neither of yours correctly describes my current beliefs. I'll also try to remember to be a bit more careful about my FAI-related comments in the future, and keep in mind that not all the readers will be familiar with my other writings.

0Epiphany12y

Thanks for listening to me. I feel better about this now.

[-]wedrifid12y-40

may at the admins' option be censored on the grounds that ... anyone talking about a proposed crime on the Internet fails forever as a criminal

I like it.

[+]ChristianKl12y-50

[+]katydee12y-110

[+]kodos9612y-190

Moderation Log

448Comments