People like Eliezer are annoyed when people suggest their rhetoric hints at violence against AI labs and researchers. But even if Eliezer & co don't advocate violence, it does seem like violence is the logical conclusion of their worldview - so why is it a taboo?
(By violence I don't necessarily mean physical violence - I mean more general efforts to disrupt AI progress, including e.g. with coordinated cyber attacks.)
I think there is some value in exploring the philosophical foundations of ethics, and LessWrong culture is often up for that sort of thing. But, it's worth saying explicitly: the taboo against violence is correct, and has strong arguments for it from a wide variety of angles. People who think their case is an exception are nearly always wrong, and nearly always make things worse.
(This does not include things that could be construed as violence but only if you stretch the definition, like supporting regulation through normal legal channels, or aggressive criticism, or lawsuits. I think those things are not taboo and would support some of them.)
Here's my objection to this: unless ethics are founded on belief in a deity, they must step from humanity. So an action that can wipe out humanity makes any discussion of ethics moot; the point is, if you don't sanction violence to prevent human extinction, when do you ever sanction it? (And I don't think it's stretching the definition to suggest that law requires violence).
(This does not include things that could be construed as violence but only if you stretch the definition, like supporting regulation through normal legal channels, or aggressive criticism, or lawsuits. I think those things are not taboo and would support some of them.)
Can you clarify on this? I think most people would agree that lawsuits do count as explicitly sanctioned violence beyond some low threshold, especially in a loser-pays jurisdiction. As in that's the intended purpose of the idea, to let the victor rely on the state's monopoly on violence instead of their private means.
It's not that violence against AI labs is a taboo... it's that violence is a taboo.
This is a commonly cited failure of deontology and in particular classical liberalism. Whether physical violence is morally justified, whether it's justified by local law, whether it's justified by international rules of war, whether it's effective, and whether it's a mechanistically understandable response from victims of harm a behaviorist perspective, are all different questions. I typically answer that most violence is ineffective and yet that the motivations can be mechanistically understood as arising from locally reasonable mechanisms of thought; mo...
Consider the following rhetorical question:
Ethical vegans are annoyed when people suggest their rhetoric hints at violence against factory farms and farmers. But even if ethical vegans don't advocate violence, it does seem like violence is the logical conclusion of their worldview - so why is it a taboo?
Do we expect the answer to this to be any different for vegans than for AI-risk worriers?
Er, yes. AI risk worriers think AI will cause human extinction . Unless they believe in God, surely all morality stems from humanity, so the extinction of the species must be the ultimate harm - and preventing it surely justifies violence (if it doesn't, then what does?)
You're assuming "the violence might or might not stop extinction, but then there will be some side-effects (that are unrelated to extinction)". But, my concrete belief is that most acts of violence you could try to commit would probably make extinction more likely, not less, because a) they wouldn't work, and b) they destroy the trust and coordination mechanisms necessary for the world to actually deal with the problem.
To spell out a concrete example: someone tries bombing an AI lab. Maybe they succeed, maybe they don't. Either way, they didn't actually stop the development of AI because other labs will still continue the work. But now, when people are considering who to listen to about AI safety, the "AI risk is high" people get lumped in with crazy terrorists and sidelined.
You can imagine an argument that goes "Violence against AI labs is justified in spite of the direct harm it does, because it would prevent progress towards AGI." I have only ever heard people say that someone else's views imply this argument, and never actually heard someone actually advance this argument sincerely; nevertheless the hypothetical argument is at least coherent.
Yudkowsky's position is that the argument above is incorrect because he denies the premise that using violence in this way would actually prevent progress towards AGI. See e.g. here and the following dialogue. (I assume he also believes in the normal reasons why clever one-time exceptions to the taboo against violence are unpersuasive.)
Well, it's clearly not true that violence would not prevent progress. Either you believe AI labs are making progress towards AGI - in which case, every day they're not working on it, because their servers have been shut down, or more horrifically, because some of their researchers have been incapacitated is a day that progress is not being made - or you think they're not making progress anyway, so why are you worried?
I addressed a general question like that in https://www.lesswrong.com/posts/p2Qq4WWQnEokgjimy/respect-chesterton-schelling-fences
Basically, guardrails exist for a reason, and you are generally not smart enough to predict the consequences of removing them. This applies to most suggestions of the form "why don't we just <do some violent thing> to make the world better". There are narrow exceptions where breaking a guardrail has actual rather than imaginary benefits, but finding them requires a lot of careful analysis and modeling.
Umm, because individual non-government-sanctioned violence is horrific, and generally results in severe punishment which prevents longer-term action. Oh, wait, that's why it's not used, not why it's taboo to even disuss.
It's taboo for discussion because serious planning for violence is a direct crime (conspiracy) itself. Don't do that. Open advocacy of violence also signals that, by your rules, it's OK for others to target you for violence if they disagree strongly enough. I don't recommend that, either (especially if you think your opponents are better at violence than you are).
Is all non-government-sanctioned violence horrific? Would you say that objectors and resistance fighters against Nazi regimes were horrific?
Because it's anti-social (in most cases; things like law enforcement are usually fine), and the only good timelines (by any metric) are pro-social.
Consider if it became like the Irish troubles. Do you think alignment gets solved in this environment? No. What you get is people creating AI war machines. And they don't bother with alignment because they are trying to get an advantage over the enemy, not benefit everyone. Everyone is incentivised to push capabilities as far as they can, except past the singularity threshold. And there's not even a disincentive for going past it, you're just neutral on it. So the dangerous bit isn't even that the AI are war machines, it's that they are unaligned.
It's a general principle that anti-social acts tend to harm utility overall due to second-order effects that wash out the short-sighted first-order effects. Alignment is an explicitly pro-social endeavor!
I think violence helps unaligned AI more than it helps aligned AI.
If the research all goes underground it will slow it down but it will also make it basically guaranteed that there's a competitive, uncoordinated transition to superintelligence.
When Eliezer proposes "turn all the GPUs to Rubik's cubes", this pivotal act I think IS outright violence. Nanotechnology doesn't work that way (something something local forces dominate). What DOES work is having nearly unlimited drones because they were manufactured by robots that made themselves exponentially, making ASI equipped parties have more industrial resources than the entire worlds capacity right now.
Whoever has "nearly unlimited drones" is a State, and is committing State Sponsored Violence which is OK. (By the international law of "whatcha gonna do about it")
So the winners of an AI race with their "aligned" allied superintelligence actually manufactured enough automated weapons to destroy everyone else's AI labs and to place the surviving human population under arrest.
That's how an AI war actually ends. If this is how it goes (and remember this is a future humans "won") this is what happens.
The amount of violence before the outcome depends on the relative resources of the warring sides.
ASI singleton case : nobody has to be killed, billions of drones using advanced technology attack everywhere on the planet at once. Decision makers are bloodlessly placed under arrest, guards are tranquilized, the drones have perfect aim so guns are shot out of hands and engines on military machines hit with small shaped charges. The only violence where humans die is in the assaults on nuclear weapons facilities, since math.
Some nukes may be fired on the territory of the nation hosting the ASI, this kills a few million tops, "depending on the breaks".
Two warring parties case, one party's ASI or industrial resources are significantly weaker : nuclear war and prolonged endless series of battles between drones. Millions or billions of humans killed as collateral damage, battlefields littered with nuclear blast craters and destroyed hardware. "Minor inconvenience" for the winning side since they have exponentially built robotics, the cleanup is rapid.
Free for all, everyone gets ASI, it's not actually all that strong in utility terms : Outcomes range from a world of international treaties similar to now and a stable equilibria or a world war that consumes the earth, most humans don't survive. Again, it's a minor inconvenience for the winners. No digital data is lost, exponentially replicated robotics mean the only long term cost is a few years to clean up.
I'd suggest reading deepmind's recent inter-org paper on model evaluation for extreme risks. What you describe as the success case I agree is necessary for success but without sufficient alignment of each person's personal asi to actually guarantee it will in fact defend against malicious and aggressive misuse of ai by others, you're just describing filling the world with loose gunpowder.
If someone thinks that violence against AI labs is bad, then they will make it a taboo because they think it is bad, and they don't want violent ideas to spread.
There are a lot of interesting discussions to be had on why one believes this category of violence to be bad, and you can argue against these perspectives in a fairly neutral-sounding, non-stressful way, quite easily, if you know how to phrase yourself well.
A lot (although not all) people are fairly open to this.
If someone thinks that violence against AI labs is good, then they probably really wouldn't want you talking about it on a publicly accessible, fairly well-known website. It's a very bad strategy from most pro-violence perspectives.
I'm going to quite strongly suggest, regardless of anyone's perspectives on this topic, that you probably shouldn't discuss it here - there are very few angles from which this could be imagined to be a good thing for any rationalism-associated person/movement. Or at least that you put a lot of thought into how you talk about it. Optics are a real and valuable thing, as annoying as that is.
Even certain styles of discussing anti-violence can come across as optically weird if you phrase yourself in certain ways.
Perhaps they prefer not to be held responsible when it happens
I try to adhere to the principle that "there are no stupid questions", but this question, if not necessarily stupid, is definitely annoying.
Do you ask the same question of opponents of climate change? Opponents of open borders? Opponents of abortion? Opponents of gun violence?
The world is full of things which are terrible, or which someone believes to be terrible. If someone, whether through action or inaction, is enabling a process that you think might kill you or cripple you or otherwise harm you, or people you care about - et cetera - then yes, violence naturally comes to mind.
But there are obvious reasons to be cautious about it, and to be cautious about talking about it. If you do it, you may end up dead or in jail. Despite your emotions, your reason may tell you that a single act of violence won't actually make any difference. You may be afraid of unleashing something that goes in a completely different direction - violence, once unleashed, has a way of doing that.
On top of that, if you're a civilized person, you don't ever want to resort to violence in the first place.
... OK, with that off my chest: if I do try to empathize with the spirit in which this question might have been asked, I imagine it as a young man's question, someone for whom the world is still their oyster, and someone who, while not an aggressive thug, is governed more by their private ethical code and their private sense of what is right and wrong, than by fear of the law or fear of social judgment or fear of unintended consequences. Willing to consider anything, and trusting their own discernment.
And then they stumble into this interesting milieu where people are really worked up about something. And the questioner, while remaining agnostic about the topic, is willing to think about it. But they notice that in all the discussion about this supposedly world-threatening matter, no one is talking about just killing the people who are the root of the problem, or blowing up their data centers, or whatever. And so the questioner says, hey guys, if this thing is really such a great danger, why aren't you brainstorming how to carry out these kind of direct actions too?
I've already provided a few reasons why one might not go down that path. But the other side of the coin is, if there are people on that path, they won't be talking about it in public. We'll just wake up one day, and the "unthinkable" will have happened, the same way that we all woke up one day and Russia had invaded Ukraine, or the ex PM of Japan had been assassinated.
Do you ask the same question of opponents of climate change? Opponents of open borders? Opponents of abortion? Opponents of gun violence?
They're not the same. None of these are extinction events; if preventing the extinction of the human race doesn't legitimise violence, what does? (And if you say nothing, does that mean you don't believe in the enforcement of laws?)
Basically, I can't see a coherent argument against violence that's not predicated either on a God, or on humanity's quest for 'truth' or ideal ethics; and the latter is obviously cut short if humans go extinct, so it wouldn't ban violence to prevent this outcome.
You write:
it does seem like violence is the logical conclusion of their worldview
It's not expected to be effective, as has been repeatedly pointed out, it's not a valid conclusion. Only state-backed law/treaty enforcement has the staying power to coerce history. The question of why it's taboo is separate, but before that there is an issue with the premise.
The assassination of Archduke Ferdinand certainly coerced history, and it wasn't state-backed. So did that of Julius Ceasar, as would have Hitler's, had it been accomplished.
This site and community generally operates on classical liberal principles, in particular a heavy focus on norms about individual acts which are forbidden. Whether that's good is up to debate; folks here are very consequentialist within some constraints. There are also consequentialist arguments for nonviolence I've heard, in particular check out critch's recent post.
Related, but, I've talked with multiple rats who, after some convincing, basically admitted, "Yeah, assuming it would actually work, I suppose I actually would push the nuclear button, but I would never admit it, because saying so would have various negative effects."
I think there is some value in exploring the philosophical foundations of ethics, and LessWrong culture is often up for that sort of thing. But, it's worth saying explicitly: the taboo against violence is correct, and has strong arguments for it from a wide variety of angles. People who think their case is an exception are nearly always wrong, and nearly always make things worse.
(This does not include things that could be construed as violence but only if you stretch the definition, like supporting regulation through normal legal channels, or aggressive criticism, or lawsuits. I think those things are not taboo and would support some of them.)
Someone or some group with objectionable moral/ethical/social positions, seen by a substantial fraction of society, can nonetheless win lawsuits and rely on the state's violent means for enforcement of the awards.
e.g. a major oil company winning a lawsuit against activists, forcing environmental degradation of some degree.
or vice versa,
e.g. nudist lifestyle and porn activists winning lawsuits against widely supported restrictions on virtual child porn, forcing a huge expansion in the effective grey area of child porn.
The losing side being punis... (read more)