Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Uncritical Supercriticality

44 Post author: Eliezer_Yudkowsky 04 December 2007 04:40PM

Followup toResist the Happy Death Spiral

Every now and then, you see people arguing over whether atheism is a "religion".  As I touched on in Purpose and Pragmatism, arguing over the meaning of a word nearly always means that you've lost track of the original question.  How might this argument arise to begin with?

An atheist is holding forth, blaming "religion" for the Inquisition, the Crusades, and various conflicts with or within Islam.  The religious one may reply, "But atheism is also a religion, because you also have beliefs about God; you believe God doesn't exist."  Then the atheist answers, "If atheism is a religion, then not collecting stamps is a hobby," and the argument begins.

Or the one may reply, "But horrors just as great were inflicted by Stalin, who was an atheist, and who suppressed churches in the name of atheism; therefore you are wrong to blame the violence on religion."  Now the atheist may be tempted to reply "No true Scotsman", saying, "Stalin's religion was Communism."  The religious one answers "If Communism is a religion, then Star Wars fandom is a government," and the argument begins.

Should a "religious" person be defined as someone who has a definite opinion about the existence of at least one God, e.g., assigning a probability lower than 10% or higher than 90% to the existence of Zeus?  Or should a "religious" person be defined as someone who has a positive opinion, say a probability higher than 90%, for the existence of at least one God?  In the former case, Stalin was "religious"; in the latter case, Stalin was "not religious".

But this is exactly the wrong way to look at the problem.  What you really want to know—what the argument was originally about—is why, at certain points in human history, large groups of people were slaughtered and tortured, ostensibly in the name of an idea.  Redefining a word won't change the facts of history one way or the other.

Communism was a complex catastrophe, and there may be no single why, no single critical link in the chain of causality.  But if I had to suggest an ur-mistake, it would be... well, I'll let God say it for me:

"If your brother, the son of your father or of your mother, or your son or daughter, or the spouse whom you embrace, or your most intimate friend, tries to secretly seduce you, saying, 'Let us go and serve other gods,' unknown to you or your ancestors before you, gods of the peoples surrounding you, whether near you or far away, anywhere throughout the world, you must not consent, you must not listen to him; you must show him no pity, you must not spare him or conceal his guilt. No, you must kill him, your hand must strike the first blow in putting him to death and the hands of the rest of the people following.  You must stone him to death, since he has tried to divert you from Yahweh your God."  (Deuteronomy 13:7-11, emphasis added)

This was likewise the rule which Stalin set for Communism, and Hitler for Nazism: if your brother tries to tell you why Marx is wrong, if your son tries to tell you the Jews are not planning world conquest, then do not debate him or set forth your own evidence; do not perform replicable experiments or examine history; but turn him in at once to the secret police.

Yesterday, I suggested that one key to resisting an affective death spiral is the principle of "burdensome details"—just remembering to question the specific details of each additional nice claim about the Great Idea.  (It's not trivial advice.  People often don't remember to do this when they're listening to a futurist sketching amazingly detailed projections about the wonders of tomorrow, let alone when they're thinking about their favorite idea ever.)  This wouldn't get rid of the halo effect, but  it would hopefully reduce the resonance to below criticality, so that one nice-sounding claim triggers less than 1.0 additional nice-sounding claims, on average.

The diametric opposite of this advice, which sends the halo effect supercritical, is when it feels wrong to argue against any positive claim about the Great Idea.  Politics is the mind-killer.  Arguments are soldiers.  Once you know which side you're on, you must support all favorable claims, and argue against all unfavorable claims.  Otherwise it's like giving aid and comfort to the enemy, or stabbing your friends in the back.

If...

  • ...you feel that contradicting someone else who makes a flawed nice claim in favor of evolution, would be giving aid and comfort to the creationists;
  • ...you feel like you get spiritual credit for each nice thing you say about God, and arguing about it would interfere with your relationship with God;
  • ...you have the distinct sense that the other people in the room will dislike you for "not supporting our troops" if you argue against the latest war;
  • ...saying anything against Communism gets you stoned to death shot;

...then the affective death spiral has gone supercritical.  It is now a Super Happy Death Spiral.

It's not religion, as such, that is the key categorization, relative to our original question:  "What makes the slaughter?"  The best distinction I've heard between "supernatural" and "naturalistic" worldviews is that a supernatural worldview asserts the existence of ontologically basic mental substances, like spirits, while a naturalistic worldview reduces mental phenomena to nonmental parts.  (Can't find original source thanks, g!)  Focusing on this as the source of the problem buys into religious exceptionalism.  Supernaturalist claims are worth distinguishing, because they always turn out to be wrong for fairly fundamental reasons.  But it's still just one kind of mistake.

An affective death spiral can nucleate around supernatural beliefs; especially monotheisms whose pinnacle is a Super Happy Agent, defined primarily by agreeing with any nice statement about it; especially meme complexes grown sophisticated enough to assert supernatural punishments for disbelief.  But the death spiral can also start around a political innovation, a charismatic leader, belief in racial destiny, or an economic hypothesis.  The lesson of history is that affective death spirals are dangerous whether or not they happen to involve supernaturalism.  Religion isn't special enough, as a class of mistake, to be the key problem.

Sam Harris came closer when he put the accusing finger on faith. If you don't place an appropriate burden of proof on each and every additional nice claim, the affective resonance gets started very easily.  Look at the poor New Agers.  Christianity developed defenses against criticism, arguing for the wonders of faith; New Agers culturally inherit the cached thought that faith is positive, but lack Christianity's exclusionary scripture to keep out competing memes.  New Agers end up in happy death spirals around stars, trees, magnets, diets, spells, unicorns...

But the affective death spiral turns much deadlier after criticism becomes a sin, or a gaffe, or a crime.  There are things in this world that are worth praising greatly, and you can't flatly say that praise beyond a certain point is forbidden.  But there is never an Idea so true that it's wrong to criticize any argument that supports it.  Never.  Never ever never for ever.  That is flat.  The vast majority of possible beliefs in a nontrivial answer space are false, and likewise, the vast majority of possible supporting arguments for a true belief are also false, and not even the happiest idea can change that.

And it is triple ultra forbidden to respond to criticism with violence.  There are a very few injunctions in the human art of rationality that have no ifs, ands, buts, or escape clauses.  This is one of them.  Bad argument gets counterargument.  Does not get bullet.  Never.  Never ever never for ever.

 

Part of the Death Spirals and the Cult Attractor subsequence of How To Actually Change Your Mind

Next post: "Evaporative Cooling of Group Beliefs"

Previous post: "Resist the Happy Death Spiral"

Comments (158)

Sort By: Old
Comment author: Scott_Scheule 04 December 2007 05:12:40PM 2 points [-]

Sam Harris came closer when he put the accusing finger on faith. If you don't place an appropriate burden of proof on each and every additional nice claim, the affective resonance gets started very easily.

How does one determine the appropriate burden of proof?

Comment author: Today 12 February 2012 01:17:51AM 0 points [-]

I would say that is when there is empirical evidence supporting the claim but, try as you might, you can't find any that falsifies it.

Comment author: Scott_Scheule 04 December 2007 05:13:29PM 0 points [-]

Apparently I left a tag open. The first paragraph is yours, whereas the second is my question.

Comment author: manuelg 04 December 2007 05:49:06PM -1 points [-]

Minor point. It is peculiar to talk about the "death of communism" when there are about as many communists in the world as there are Christians.

"Death of the _Purported_ Worldwide Worker's Communist Revolution" is closer to the truth (and a mouthful).

How about "Death of Worldwide Revolutionary Communism"?

Comment author: anonymous7 04 December 2007 05:51:47PM 0 points [-]

The problem with Communism is timing. In the future (if we survive the next century) there will be enough technological progression to create essential Communism (no-one needs to work, everyone will have necessary resources to live incredible lives and so forth). Of course, we won't call it Communism.

Comment author: Tiiba2 04 December 2007 06:14:03PM 0 points [-]

Haven't had a post this good in a while. With immediate application,too.

Comment author: michael_vassar3 04 December 2007 06:20:39PM 19 points [-]

Err... did that post end up dying in a free speech happy death spiral?

Especially odd from a person who believes in the probable possibility of humanly irresistible bad arguments as a reason for not AI boxing. If there are minds that we can't let exist because they would make bad arguments that we would find persuasive this seems terribly close, from an aggregative utilitarian standpoint, to killing them.

I'm not an expert in the Rwandan genocide, but it's my impression that to a substantial extent the people behind it basically just made arguments (bad ones, of a primarily ad-hominem form like "Tutsis are like cockroaches") for killing them and people who listened to those arguments on the radio went along with it. At least with the benefit of hindsight I am reluctant to say that the people promoting that genocide should have been stopped forcibly. Similarly, it's my impression that Charles Manson didn't personally kill anyone. He merely told his followers ridiculous stories of what the likely results of their killing certain people would be.

It would be nice if, as Socrates claimed, a bad argument cannot defeat a good one, but if that was true we wouldn't need to overcome bias. With respect to our own biases, hopefully careful thought and study of psychology is the only tool we will ever need to overcome them, but with respect to the biases of others it would be terribly biased to never consider the possibility that other tools are necessary. We can find good heuristics, like "don't violently suppress anyone who isn't actively promoting violence", but sadly violence isn't a basic ontological category, so we can't cleanly divide the world into violent and non-violent actions, no into statements that promote or don't promote some conclusion (in the context of what goal system?).

Comment author: Eliezer_Yudkowsky 04 December 2007 06:37:30PM 15 points [-]

Especially odd from a person who believes in the probable possibility of humanly irresistible bad arguments as a reason for not AI boxing. If there are minds that we can't let exist because they would make bad arguments that we would find persuasive this seems terribly close, from an aggregative utilitarian standpoint, to killing them.

Fine, let me rephrase: in the human art of rationality there's a flat law against meeting arguments with violence, anywhere in the human world. In the superintelligent domain, as you say, violence is not an ontological category and there is no firm line between persuading someone with a bad argument and reprogramming their brain with nanomachines. In our world there is a firm line, however.

Let me put it this way: If you can invent a bullet that, regardless of how it is fired, or who fires it, only hits people who emit untrue statements, then you can try to use bullets as part of a Bayesian analysis. Until then, you really ought to consider the possibility of the other guy shooting back, no matter how right you are or how wrong they are, and ask whether you want to start down that road.

If the other guy shoots first, of course, that's a whole different story that has nothing to do with free speech.

Comment author: Matthew2 04 December 2007 06:42:59PM 0 points [-]

So what is your response to someone like Hitler? Assuming the thug won't listen? Die? Run? I mean before the AGI goes "phoom".

Comment author: g 04 December 2007 06:54:58PM 4 points [-]

Eliezer, I first saw the distinction between "natural" and "supernatural" made the way you describe in something by Richard Carrier. It was probably a blog entry from 2007-01, which points back to a couple of his earlier writings. I had a quick look at the 2003 one, and it mentions a few antecedents.

Comment author: Tom3 04 December 2007 06:56:37PM 2 points [-]

anonymous:

"In the future (if we survive the next century) there will be enough technological progression to create essential Communism (no-one needs to work, everyone will have necessary resources to live incredible lives and so forth)."

-10 points for confusing means with ends.

From the article:

"[...]there is never an Idea so true that it's wrong to criticize any argument that supports it."

Or make jokes about it? Having a sense of humour ought to be mentioned as a primary piece of equipment in the Bias-Buster's toolkit. It's easy and fun! After all, a defining feature of True Believers is that they lack a sense of irony.

Comment author: Peter_de_Blanc 04 December 2007 06:58:35PM 2 points [-]

Eli, you said:

In the superintelligent domain, as you say, violence is not an ontological category and there is no firm line between persuading someone with a bad argument and reprogramming their brain with nanomachines. In our world there is a firm line, however.

I don't think there is such a firm line. I think argument shades smoothly into cult brainwashing techniques.

Comment author: michael_vassar3 04 December 2007 08:09:15PM 3 points [-]

Peter: It seems to me that we can draw a firm line, but on one side sits our very strictest most careful thought in the spirit of inquiry and on the other sits everything remotely aimed at influencing others, from legal argument to scientific defense of a position to advertising to flirtation to music (at least lyrical music) to conversation using body language and tones of voice to cult brainwashing techniques and protest rallies etc. It's very clear that we can't live entirely to one side of that line, or if we can, that we can only live on the side that contains, well, life, and also, sadly, violence.

Comment author: Tom_McCabe2 04 December 2007 08:37:31PM 9 points [-]

"Bad argument gets counterargument. Does not get bullet. Never. Never ever never for ever."

What about knowledge which is actually *dangerous*, eg., the Utterly Convincing and Irresistible Five-Minute Seminar on Why We Should Build a UFAI, with highly detailed technical instructions.

Comment author: anonymous7 04 December 2007 08:41:06PM 0 points [-]

Tom, I did not confuse ends and means.

Comment author: TGGP4 04 December 2007 08:46:56PM 3 points [-]

there are about as many communists in the world as there are Christians. Really? There are a lot of Christians. From what I've read, virtually nobody in China is a communist now, just as people had stopped believing in the last days of the Soviet Union. In North Korea or among the rebels of Nepal there are still true-believers, but I don't think there are as many as there are Christians.

In general I like having a norm against using force when people make bad arguments. I deplore the anti-fascist fascists who seem to be the primary enemies of free speech today. At the same time I recognize that in some situations it could hypothetically be the case that free speech leads to bad outcomes, in which case I'd be alright with restricting it. I think such cases would be fantastically rare and would likely only occur during a civil war (a category I don't consider wars of secession to be members of). I recognize though that in normal situations giving a directive/command as opposed to an argument for something should be treated as solicitation of an act. Stephan Kinsella discusses that in Causality and Aggression.

Comment author: Psy-Kosh 04 December 2007 09:05:41PM 2 points [-]

Incidentally, I've taken to using the term "afaithist" for myself rather than "atheist" largely due to above mentioned issues. I'm not all that concerned so much about various religious beliefs rather than the notion of the virtue of non rational/anti rational belief, including various "must not question" flavors. Questions like existance of god/etc etc are almost incidental, questions of "mere" (eheh) fact.

Tom: If there was such a convincing eminar, perhaps it contains such a convincing argument that it's genunitely correct. Modify it to "Utterly Convincing and Irresistable Five-Minute Brainwashing Seminar On Why....." :)

Comment author: buybuydandavis 27 October 2011 09:37:44AM 0 points [-]

Afaithist. That's very good. I like it.

Comment author: Larry_D'Anna 04 December 2007 09:53:58PM 1 point [-]

no, anonymous. The problem with communism is that it's coercive and tyrannical. A super-duper welfare state is not the same as communism. Especially as productivity goes up. The difference being: under a welfare state you are taxed a portion of what you have, and some of that goes to the poor. Under communism you are essentially owned by the state. The state can tell you when to work, what to work on, and how many hours. The state tells you what you can or cannot buy, because the state decides what will or will not be produced.

Whatever you think about welfare states, communism is something else entirely.

Comment author: Tom_McCabe2 04 December 2007 11:29:27PM 0 points [-]

"Tom: If there was such a convincing eminar, perhaps it contains such a convincing argument that it's genunitely correct. Modify it to "Utterly Convincing and Irresistable Five-Minute Brainwashing Seminar On Why....." :)"

It was just an example; it was deliberately chosen to be as extreme as possible, to avoid grey-area questions. No human, so far as I can determine, has the intelligence to actually produce such a thing. Many less-extreme examples of this abound, eg., what should we do with blueprints for nuclear weapons? What about genetic databases which include deadly viruses? What about genetic databases which are .1% deadly viruses and 99.9% life-saving medical research? And on and on it goes.

Comment author: Caledonian2 05 December 2007 12:22:16AM 0 points [-]

in the human art of rationality there's a flat law against meeting arguments with violence, anywhere in the human world

No. You're confusing rationality with your own received ethical value system. Violence is both an appropriate and frequently necessary response to all sorts of arguments.

Comment author: taryneast 15 February 2011 01:31:34PM *  2 points [-]

A strong claim but without any evidence to back it up. Perhaps you could at least give some examples of arguments for which the necessary response is violence.

Comment author: wedrifid 15 February 2011 01:59:10PM 3 points [-]

Perhaps you could at least give some examples of arguments for which the necessary response is violence.

Possibly, but only if you roll a natural 20 on your necromancy check. You are replying to a comment that was posted in 2007 - and on a different blog! Perhaps not the ideal place to play the "my position is the default - you are the one who needs to supply all the evidence!" game. Or, then again, perhaps it is the ideal place!

A strong claim but without any evidence to back it up.

It is not even a question for which demanding evidence makes sense - at least without specifying more clearly what kind of observations of the world you are considering. One could assume that you mean "give me evidence that the consequences of responding to arguments with violence can positive" - but then you have already lost to Caledonian's position. When you are looking at consequences, "argument" and "violence" are just two different kinds of power. Occasionally the latter is to be preferred to the former.

The only way "there's a flat law against meeting arguments with violence, anywhere in the human world" was going to hold was if it stayed purely in the ideological realm. And a Traditional Rationality ideology realm more than a Bayesian Rationality one. "Arguments" can, at times, be a greater epistemic rationality violation than mere violence.

Comment author: taryneast 15 February 2011 07:26:30PM *  4 points [-]

Possibly, but only if you roll a natural 20 on your necromancy check.

:) I did know that I was responding to an ancient response... but I had thought that Caledonian may still be lurking about this site...

In a way my response was more to point out the problem with what he said - than to actually request a specific response from him. If somebody else came along later and happened to agree with Caledonian, they might point out evidence that would support his claim... thus I figured it was worth posting anyway.

We keep being told to comment regardless of how old the posts are... and this is why.

One could assume that you mean "give me evidence that the consequences of responding to arguments with violence can positive"

Nah - that's the wrong tack. I'm sure there are things where violence could be a positive response. But the claim made was that there are things for which the necessary response is violence... as though for certain situations, only violence will work.

Perhaps there are such situations.. but Caledonian did not even give examples, let alone evidence to support his claim... Thus my reaction.

I'd argue that there are vanishingly few situations in which the only possible solution is violence... but, as I stated, would welcome evidence/discussion to the contrary.

Comment author: ThrustVectoring 04 April 2011 08:23:09AM 2 points [-]

This might be stretching the definition of an "argument", but I think there's a class of speech that must be dealt with by violence. The key identifier of this class is that there is a time-critical danger from third parties accepting the argument.

In other words, its not so much violence used to prevent Alice from trying to convince you that the sky is red, but violence used to prevent Alice from trying to convince Bob to participate in a lynching.

Comment author: taryneast 04 April 2011 12:32:18PM 1 point [-]

I'd disagree that violence was the only option in that case. I think the best option might be to spirit away the potential lynchee. If they've already got him strung up - then firing a shot in the air, followed by harsh words from the local law-maker would be the next option... violence is still an option, but not the only one, and not necessarily the first port of call.

I think it's quite rare for violence to be the only option available.

Comment author: TheOtherDave 04 April 2011 12:56:14PM 6 points [-]

Drawing a sharp distinction like this between violence and the implied threat of violence (e.g., firing weapons and "harsh words" and the invoking of authority backed by force) is problematic. The efficacy of the latter depends on the former; a law-maker known to be reliably nonviolent firing a harmless noisemaker would be far less effective.

Comment author: Larks 10 April 2011 10:17:20PM 1 point [-]

I had thought that Caledonian may still be lurking about this site...

Actually, I think he got banned - if you look at his last comments, he certainly thought it likely that he was going to be.

Comment author: taryneast 11 April 2011 12:15:33PM 1 point [-]

Yeah - could be. I also read a comment from EY saying that it was specifically his vote keeping him unbanned.

I've read other comments from EY that seem to suggest that Caledonian was being kept around as a kind of troll-in-residence.... including one that seemed to indicate that EY used responses to Caledonian to determine whether or not he'd got his point across well enough :)

Comment author: TheOtherDave 11 April 2011 12:58:19PM 1 point [-]

Interesting. I don't remember those, but I do remember several discussions where EY wanted to ban Caledonian and various other people talked him out of it.

Comment author: jhuffman 21 April 2011 03:24:21PM 1 point [-]

You may be right, but I wanted to point out that I think that violent repression of speech is an accepted part of common law and is an available remedy in the United States and other countries based on English common law. Civil actions related to libel and slander ultimately carry a threat of violence by the state in recovering judgments found against someone in a civil court; you never see it actually happen, and often judgments are not collected at all but it is completely possible you could be found in contempt or have a lien against property that eventually results in an arrest warrant.

Comment author: Will_Sawin 21 May 2011 02:55:43AM 0 points [-]

My understanding is that it's supposed to be a "Bayesian Rationality on leaky hardware" thing. This makes finding evidence for and against very subtle, because you have to come up with some kind of reference class that's objective in a certain hard-to-define way.

But some kind of argumentation is necessary and has some chance of working.

Comment author: manuelg 05 December 2007 12:31:36AM 0 points [-]

> From what I've read, virtually nobody in China is a communist now, just as people had stopped believing in the last days of the Soviet Union. In North Korea or among the rebels of Nepal there are still true-believers, but I don't think there are as many as there are Christians.

I find it useful to distinguish between the Chinese and the Swedish. I call the Chinese form of government "communism", and I call the Swedish form of government "socialism". If they are all sub-tribes of "Canadians" to you, then you don't prize distinction as much I do.

There are certainly more "self-reported communists" than there are "humans whose daily actions are informed by the example of Jesus Christ".

...All I need are are a few dozen "self-repored communists" to prove that...

Comment author: Rolf_Nelson2 05 December 2007 01:31:06AM 1 point [-]

And it is triple ultra forbidden to respond with violence.

I agree. However, here are my minority beliefs on the topic: unless you use Philosophical Majoritarianism, or some other framework where you consider yourself as part of an ensemble of fallible human beings, it's fairly hard to conclusively demonstrate the validity of this rule, or indeed to draw any accurate conclusions about what to do in these cases.

If I consider my memories and my current beliefs in the abstract, as not a priori less infallible than anyone else's, a "no exceptions to Freedom to Dissent" policy follows naturally. But if I, instead, always model my last ten minutes of thought as part of a privileged, infallible Bayesian process, then the simplest conclusion is that I have a right, and even a moral duty, to mete out political punishment as I see fit. (You're also logically required, if the latter is your world-view, to eschew stock-market index funds in favor of placing diverse InTrade bets with your savings; finally, you're also required, if a betting market ever opens up where people can bet on the consequences of mathematical proofs, to bet against the greatest mathematicians in the world, anytime you disagree with them on whether a proof is valid or not.)

Comment author: Eliezer_Yudkowsky 05 December 2007 03:02:54AM 3 points [-]

A rule of human rationality becomes flat when the probability of falsely perceiving an exception to the rule, vastly exceeds the probability of ending up in a real-world situation where it genuinely makes sense to violate the rule. Relative to modern Earth which includes many violent people and many difficult ethical dilemmas, but does not include the superbeing Omega credibly threatening to launch a black hole at the Sun if you don't shoot the next three Girl Scouts who try to sell you cookies.

I think a lot of the commenters to this thread are also missing the counterintuitive idea that once you fire a gun, the target or their survivors may shoot back.

Comment author: michael_vassar3 05 December 2007 03:19:02AM 3 points [-]

Isn't the probability of ending up in a real world situation where the entire world is in terrible danger and only you can save it vastly smaller than that of falsely perceiving such a situation? Despite that, I'm glad Petrov made his decision. Expected costs and benefits have to be considered, not just probabilities, but then you are back in normal decision theory or at least normal but not yet invented "decision theory for biased finite agents".

Comment author: Nominull2 05 December 2007 05:15:41AM 8 points [-]

Not murdering people for criticizing your beliefs is, at the very least, a useful heuristic.

Comment author: Eliezer_Yudkowsky 05 December 2007 05:24:59AM 4 points [-]

Isn't the probability of ending up in a real world situation where the entire world is in terrible danger and only you can save it vastly smaller than that of falsely perceiving such a situation? Despite that, I'm glad Petrov made his decision.

Fair enough. s/probability of/expected utilities associated with/

But you can still end up with a "flat" rule for the human art of rationality, when the expected negative utilities associated from biased decisions that "the end justifies the means, in just this one case here", exceeds the expected positive utilities from cases where the universe really does end up a better place from shooting someone who makes an argument you don't like after taking all side effects into account including encouragement of similar behavior by others.

Remember, human targets shoot back. Since bullets are not even probabilistically more likely to hit when fired at a human target who has just made false statements as opposed to true statements, it's very difficult to see how a social decision process can be made more rational by introducing bullets into it.

Comment author: michael_vassar3 05 December 2007 05:25:28AM 1 point [-]

Agreed Nominull, spectacularly useful. Definitely the sort of heuristic one would sensibly like to promote.

Rolf: It seems to me that you are trying to assert that it is normative for agents to behave in a certain manner *because the agents you are addressing are presumably non-normative*. The trouble is, using that strategy you guarantee no normative agents. The non-normative agents are not corrected by adopting your strategy, as it only mitigates their irrationalities, while any normative agents are adopting an inappropriate strategy. You can never choose soundly by assuming the processes generating your choices not to be sound. Looks to me like we need a more coherent concept of normativity, preferably one without supernatural agents. With flawed agents, all we can rigorously say is that they do what they do. What room for "should" once you have assumed them flawed and what room even for "could" once you have decided to treat them as causal systems.

Comment author: J_Thomas 05 December 2007 06:58:58AM 0 points [-]

in the human art of rationality there's a flat law against meeting arguments with violence, anywhere in the human world

"No. You're confusing rationality with your own received ethical value system. Violence is both an appropriate and frequently necessary response to all sorts of arguments."

I want to note that Buzz Aldrin, the second man to set foot on the moon, famously encountered a man who denied that humans have ever gone to the moon but that the videos of Buzz on the moon were filmed in arizona. Buzz's response when the man presented his arguments was to sock him in the jaw.

The science fiction writer John Barnes, who collaborated with Aldrin on a couple of science fiction novels, has since written several novels in which the appealing protagonist argues that the only appropriate response to some arguments is a good swift sock in the jaw. His protagonists do so, with good results.

Millions of impressionable young science fiction readers are influenced by these novels.

If you met John Barnes and he argued that he's doing the right thing, would it be appropriate to sock him in the jaw?

Barnes is 53 years old but has been doing martial arts for something like 30 years. Would that influence your choice?

Should you let the moral value of initiating violence depend on whether or not you win?

If it's right to physically attack somebody who disagrees with you provided you win but wrong when you lose, what about when it's a ten year old girl who makes an argument you can't answer except with violence?

Comment author: aphyer 09 October 2011 04:51:33AM 0 points [-]

I realize that this comment has been up for a long time, but just in defense of Buzz Aldrin: the punch was less in response to the man claiming that he was wrong, and more in response to the man being verbally abusive (don't believe everything you hear, search on Google for Buzz Aldrin Punch and you can get a video for yourselves.) There's a difference between violence being the appropriate response to reasoned argument and violence being the appropriate response to abuse/someone else's violence/etc.

Comment author: Unknown 05 December 2007 07:30:36AM 1 point [-]

In this case it seems that Eliezer is a bit biased toward defending his stated position, despite the fact that it is entirely obvious that his "flat rule" is in fact a leaky generalization.

For example, he keeps mentioning consequences that result from the response of the person attacked or the imitation of others. These consequences will not follow in every case. There will be no such consequences when no one (including the person attacked) will ever find out that one has responded to an argument with violence.

One can easily think of many theoretically possible circumstances (not involving superintelligence) in which one can prevent immense evils and bring about immense goods by responding to an argument with violence, and yet satisfying the condition above, that no one will ever find out.

It is true that such circumstances are not particularly probable, yet they might well be quite recognizable if they actually happened. Thus, there can be no such flat rule of rationality.

Comment author: martin_lb 05 December 2007 12:23:08PM 0 points [-]

It seems to me Eleizer has arrived at a line of arguing that mirrors buddhusm and other similar systems. People are attached to certain ideas, concepts, beliefsystems etc, and when two opposing ideas clash together the result is killing, and a destructive spiral. The challenge is to transcend the situation, by being able to keep the mind cold when the rest of society goes amok. Unfortunately while scientists are good at discribing a situation, when it comes to giving normative advice they are mostly useless.

Another thing, faith is so often brought up as a reason for war and massmurder. Equally important imo are other caracteristics of the human psyche like fear, frustration, humiliation and hatred. Just consider why do (or did) so many americans support the senseless attacs on Afganistan and Irak? I would say more beacause they are scared shitless that some arabs might blow up their neighborhod rather than their blind belief in their leader.

Comment author: Stuart_Armstrong 05 December 2007 01:08:35PM 0 points [-]

But you can still end up with a "flat" rule for the human art of rationality, when the expected negative utilities associated from biased decisions that "the end justifies the means, in just this one case here", exceeds the expected positive utilities from cases where the universe really does end up a better place from shooting someone who makes an argument you don't like after taking all side effects into account including encouragement of similar behavior by others.

That might deserve a post of it own...

There will be no such consequences when no one (including the person attacked) will ever find out that one has responded to an argument with violence.

There are also effects on the attacker - specifically, it may make it easier for them to attack again. It will change their attitude on many issues. You'd need a murder-suicide to pull this off properly...

One can easily think of many theoretically possible circumstances (not involving superintelligence) in which one can prevent immense evils and bring about immense goods by responding to an argument with violence, and yet satisfying the condition above, that no one will ever find out.

Indeed. Eliezer's rule cannot be a flat-out rule unless it's the only flat-out rule of moral conduct - otherwise it's nearly certain it can be contradicted by one of the other flat-out rules (barring strange specific moral rule constructions that are used by nobody).

But if we take "flat-out" to mean "This rule has more weight behind it than you can imagine. Much more. You're not there yet - you can't yet conceive how bad the consequences of violating this rule will be", then it's acceptable.

Comment author: Unknown 05 December 2007 01:26:58PM 0 points [-]

"You'd need a murder-suicide to pull this off properly..."

Reminds me of Agatha Christie's "Curtain". Of course this is fictional evidence, and in any case I was thinking of more obviously justified cases.

Comment author: DaCracka 05 December 2007 03:17:32PM -1 points [-]

You can think of reasons to be violent, you can think of the good that violence might create, but consider this:

The only human being who is remembered as being completely good because he shot someone was Hitler, when he shot himself.

The list of possitive changes accomplished in the REFUSAL to shoot anyone is much longer.

I don't believe violence can ever have a positive effect, except when used to defend against greater violence.

In argument, short of the entirely impossible situation where an abominable idea is irrestable to everyone else, (and assuming that you are the one person capable of resisting it...) having a 99.9999999999% probability assigned that non-violence is preferrable by a vast margin, in almost every possible situation, would be a good guide line for even the strictest rationalist.

Comment author: DaCracka 05 December 2007 03:21:53PM 0 points [-]

Let me rephrase: Hitler's action (suicide) was for the good. Not he as a human being, or pretty much anything else he did. (With the exception of painting, those weren't bad.) I really should proofread this before I come off as saying something completely different.

Comment author: Eliezer_Yudkowsky 05 December 2007 05:37:49PM 15 points [-]

There are plenty of situations where violence is the correct answer. There are even situations where being the first to initiate violence is the correct answer, for example, to establish a property-ownership system and enforce against anyone being able to wander in and eat the crops you grew, even if they don't use violence before eating.

However, in real life, initiation of violence is never the correct answer to a verbal argument you don't like. Anyone can "imagine" exceptions to the rule, involving definite knowledge that an argument persuading other people is wrong, and (more difficult) absolute knowledge of the consequences, and (most difficult) themselves being the only people in the world who will ever pick up a gun. Except that it's easy to forget these as conditions, if you imagine in a naively realistic way - postulate a "wrong argument" instead of your own belief that an argument is wrong, postulate "I shoot them and that makes the problem go away" instead of your own belief that these are the consequences, and just not think about anyone else being inspired to follow the same rule. Real ethical rules, however, have to apply in the case of states of knowledge, rather than states of reality. So don't tell me about situations in which it is appropriate to respond to an argument with violence. Tell me about realistically obtainable states of belief in which it is appropriate to respond to an argument with violence.

Comment author: michaelcurzi 11 April 2011 04:57:53AM 0 points [-]

I've found this response to be incredibly useful in other discussions of morality. I hadn't found it formulated elsewhere, and had been looking for something like it for a long time.

Comment author: George_Weinberg2 05 December 2007 08:59:01PM 0 points [-]

It seems to me that normative statements like "let us go and serve other gods" aren't really something you can have a rational debate about. The question comes down to "which do you love more, your god or me", and the answer should always be "God"... according to God.

Similarly, one could have a rational debate about whether a command economy will outperform a market economy or vice versa (although the empirical evidence seems pretty one-sided), but a statement like "all people ought to be socially and economically equal" seems like something that just has to be accepted or rejected.

Comment author: J_Thomas 05 December 2007 09:07:05PM -3 points [-]

_So don't tell me about situations in which it is appropriate to respond to an argument with violence. Tell me about realistically obtainable states of belief in which it is appropriate to respond to an argument with violence._

I don't exactly agree with this, but I can see it as a social signal. As Buzz Aldrin and John Barnes seem to express it, when somebody makes his argument that you utterly despise, you hit him to show that you refuse to engage in rational argument with him. He is your enemy and beyond the pale and his position is not one that you consider open to rational debate.

So for example a zionist might respond this way to someone who argues that israel/palestine should become one democratic nation under one-person/one-vote. It would mean the destruction of the state of israel, it would mean that israelis would become a minority in their own country. A zionist could respond with words, something like "I don't need logical arguments for why israel should exist. Israel exists because people like me are ready and willing to kill anybody who tries to destroy her.". But a good swift sock in the jaw says the same thing more forcefully, without actually killing anybody. Ideally you knock him out and he falls down and hits his head on the floor, and when he wakes up he will be a chastened antisemite, a subdued antisemite, a far more submissive antisemite. He will not annoy you with logical argument.

Similar treatment might be effective against communists, pro-abortionists, and liberals generally. Logical argument can only carry you so far; at some point you get to principles that you accept because of who you are. You can't expect everyone to accept those same principles because they are not you. Some people accept the principle "You should not sock somebody in the jaw just because he disagrees with you" and some do not.

How we get along in social conversation with people who disagree with us says a lot about us. If you're having a civil conversation with a rapist, or a serial killer, or a republican, how do you handle yourself? Should you always wait for him to strike the first blow, or is it ever appropriate to cold-cock him with no warning?

Comment author: George_Weinberg2 05 December 2007 09:14:52PM 3 points [-]

If you met John Barnes and he argued that he's doing the right thing, would it be appropriate to sock him in the jaw?

No, because the statement that "the only appropriate response to some arguments is a good swift sock in the jaw" is not itself one of the arguments whose appropriate response is a sock in the jaw. There may or may not be any such arguments, but socking him in the jaw is admitting that he is fundamentally right. Of course, it might be appropriate to sock him for some other reason :-)

One can argue that Buzz Aldrin had a special right to sock the guy that you or I would not. To me, claiming the moon landing was faked is just an absurd statement. Saying it in front of Buzz is unjustifiably calling the man a fraud and a liar. Buzz shouldn't have to put up with that kind of crap.

Comment author: J_Thomas 05 December 2007 09:39:26PM 0 points [-]

GW, to what extent should we treat people as we want them to treat us, and to what extent should we treat them the way they say is right and the way they treat others?

Sometimes it's polite to treat other people by their own standards, and it isn't an admission that their way is right and ours is wrong.

Comment author: Eliezer_Yudkowsky 05 December 2007 09:47:37PM 7 points [-]

J Thomas: Ideally you knock him out and he falls down and hits his head on the floor, and when he wakes up he will be a chastened antisemite, a subdued antisemite, a far more submissive antisemite. He will not annoy you with logical argument.

Gosh, I hope no one ever tries anything similar on a Jew.

Comment author: Chip_Smith 05 December 2007 09:55:05PM 1 point [-]

TGGP writes:

"I recognize that in some situations it could hypothetically be the case that free speech leads to bad outcomes, in which case I'd be alright with restricting it. I think such cases would be fantastically rare..."

What about the "Werther Effect"? Journalism guidelines are drafted on the assumption that it is real, and browsing through PubMed suggests that the evidence is strong enough.

So, if imitative suicide is facilitated through art or media stimuli in predictable ways, isn't the empirical question as to whether there are "bad consequence of free speech" answered, with the reality being more prosaic than fantastically rare?

Unless you don't think suicide is a bad thing, I suppose.

Comment author: TGGP4 05 December 2007 10:40:01PM 0 points [-]

What about the "Werther Effect"? I'm not really that bothered by a bunch of people I don't know killing themselves. It's your life to make or take.

Unless you don't think suicide is a bad thing, I suppose. I think my more apathetic attitude toward human life separates me from transhumanists/immortalists. I discuss that a bit here. I'm thinking more along the lines of violent totalitarian ideologies that have a reasonable chance of taking over and really screwing things up.

Comment author: Tom3 06 December 2007 02:12:05AM 1 point [-]

You can see the Buzz Aldrin punch on Youtube.

I heard he also roundhouse kicked a holocaust denier through a plate glass window and karate chopped a 9/11 truther in the balls.

Comment author: DaCracka 06 December 2007 02:33:19PM 0 points [-]

dutz, as paintings, yes, they weren't any good. But still, much better than genocide.

Violence may convince your opponent it isn't worth arguing with you. But it will convince your audience that you're an emotional, impulsive, irrational person, no matter how right you were.

People can see someone as less than human. Until they see the getting beaten with fire hoses, and then pity sinks in.

I think in the original context, Eliezer was talking about violence commited by a society/sect/police force against an individual.

I happen to believe a swift punch in the jaw is justified in rare cases. But I can show you a few people who think beating an uppity woman is the best way to put her in her place.

You have to draw the line somewhere, and I think Buzz would agree. Sometimes, you're going to have to step over that line, so let's put it as far back as we can.

Comment author: gutzperson 06 December 2007 03:29:32PM -2 points [-]

DaCracka: I think these are two issues related in a different way. His paintings were not better than genocide. This is like saying butter is better than a smack in the face. This is kind of illogical. Though, if his paintings would have been better there would have been a chance to avoid this genocide, because the Academy would have accepted him and he might have become a painter instead of a dictator. About the violence thing. I agree nobody should react with violence to an argument. There are people out there who do so. They do it because they are either frustrated or they have not learnt to discuss, or somebody has taught them to do so, or their only language is violence. It might be in the genes or just lack of education or a social dysfunction ‌. Some people, by the way, feel better, if they shoot the alien. Some people feel better if they use violence. Some people batter their kids and spouses because they did answer back or the soup was not cooked properly. Some people are inherently violent. They are ‘anger’ machines. It gives them a kick. About violence and society. What do we define by violence? Do we also define intrusion in our personal sphere, psychological re-programming, etc. as violent activities? Capitalism can be seen as violent and intrusive. Globalisation, the forceful opening of new markets, the imposing of certain consumer and management phraseology on whole groups, the creation of seemingly unnecessary needs and obsessive consumerism. There is a whole generation of managers with certain speech and thinking patterns, they seem to be the forerunners of limited futurist AI. As posted by others, sometimes violence is the only way of avoiding even greater violence and injustice. What about the Resistance in countries that were occupied by Nazi Germany? I think that their violence was necessary violence.

Comment author: J_Thomas 06 December 2007 03:45:46PM 1 point [-]

"But there is never an Idea so true that it's wrong to criticize any argument that supports it. Never. Never ever never for ever."

Was it wrong for the guy who thought Buzz Aldrin helped fake the moon landing to present his arguments to Buzz?

One of the hungarian Manhattan-project physicists had a slogan that went "It is not enough to be rude, one must also be wrong." When it comes time to decide whether to answer a verbal argument with violence, does it matter whether the argument is wrong, or is it enough to be rude?

Comment author: DaCracka 06 December 2007 04:34:24PM 0 points [-]

guts, I would prefer butter to a slap in the face anyday. I'm sure you would, too.

The point I was making about the paintings, (in the tradition of the late Mr. Vonnegut,) is that Hitler was a person. Being a person, he should've stuck to painting, rather than violence. We should encourage more video games where people make art rather than shooting things. We should be less upset about children seeing naked people and more upset about them seeing dead ones.

In terms of a punch in the jaw:

We'd all agree that beating a child is wrong, and that Mike Tyson isn't a rationalist, on any level. You don't win an argument with a punch in the jaw, you end discussion. I've taken a beating or two for refusing to alter my beliefs, and it didn't influence my idea any. I'm sure the arm chair theorist took the punch in the jaw as confirmation that Mr. Aldrin was a mongoloid, incapable of operation of anything more complex than a tricycle.

A punch in the jaw is a reaction, and I'm sure Mr. Aldrin didn't debate the instinct. His greatest accomplishment was being questioned by an arm chair theorist. A punch in the jaw was, in hindsight, not a great action worthy of applause, but certainly understandable.

Again, Eliezer's original point was talking about large groups, (Catholics, Nazi's, Communists, Puritans, Islamic extremists... etc) committing violence, not a sock in the jaw, but a gas chamber. I still agree with him that if you have to kill and torture to defend your idea, it's probably not a good idea in the first place, or the execution isn't working out well.

JT, I think someone who's wrong, stupid, and rude deserves more sympathy than someone who's calm, open-minded, and polite. The nice person probably has more friends, and a better relationship with his family. The arrogant tend to be terribly unsatisfied, and feel inferior, so they overcompensate.

A punch in the jaw? How about a hug? Or a respectful handshake? Are you any better than the rude and uninformed if your first reaction is to wonder if you should hit them in the face?

Comment author: J_Thomas2 06 December 2007 09:05:00PM 0 points [-]

That looks like a verbal argument to me. Kind of bare without any supporting evidence, but he might have been about to provide supporting evidence. Hard to tell what he was about to do.

Comment author: TGGP2 06 December 2007 11:45:00PM 1 point [-]

About violence and society. What do we define by violence? Do we also define intrusion in our personal sphere, psychological re-programming, etc. as violent activities?
You should read Randall Collins.

What about the Resistance in countries that were occupied by Nazi Germany?
Did they actually accomplish anything? I think it was the violence of the opposing armies that actually ended Nazi occupation.

Comment author: gutzperson 07 December 2007 11:06:00AM 0 points [-]

Thanks for pointing out Randall Collins.
Resistance certainly achieved something. I mentioned it as an example for 'justified' violence.

Comment author: TGGP2 08 December 2007 04:34:00AM 0 points [-]

What did the Resistance accomplish? I already stated that it seemed to me that it was the opposing armies that got rid of the Nazis. If you disagree on that or have something else you think they did, state it.

Comment author: J_Thomas2 08 December 2007 08:55:00AM 0 points [-]

The Resistance pinned down occupation troops that otherwise would have been available to fight opposing armies.

Anyway, it's different committing violence against people who kill you if they catch you disobeying them, versus committing violence against people who are only presenting a verbal argument. Some of us take the moral stand that it's wrong to hurt people just for what they say, while others of us figure that the practical thing is to stop bad stuff at whatever stage is most effective.

_About violence and society. What do we define by violence? Do we also define intrusion in our personal sphere, psychological re-programming, etc. as violent activities?_

Gutzperson, if by "intrusion into our personal sphere" you mean saying things in our presence we don't want to hear, I'd have to say that isn't violence. If it means breaking down our doors and pointing guns at us, that comes a lot closer.

Similarly for psychological reprogramming. If it involves coercion where you give people intense negative reinforcement -- electric shocks, beatings, sleep deprivation, etc -- then that pretty much includes violence. If it's just telling them things they aren't psychologically ready to handle, I tend to think not although it's maybe a gray area. People ought to be ready to handle anything anybody says to them. But sometimes they aren't. Do we have a responsibility to respect other people's fragile mental stability by never saying anything that might unsettle them?

Comment author: Hul-Gil 21 May 2011 02:35:17AM 0 points [-]

while others of us figure that the practical thing is to stop bad stuff at whatever stage is most effective.

You're only stopping bad stuff if it's something like a threat - in other words, a declaration of intent. I can't imagine thinking it is appropriate to "sock" somebody for a dissenting opinion. Surely this is how the Inquisition, or Stalin's U.S.S.R, was justified.

I guess you wouldn't complain if I hit you for expressing these opinions.

(Woo, necromancy!)

Comment author: Nastunya 08 December 2007 10:07:00PM 0 points [-]

Eliezer, your post appears to at least in part be animated by a frustration with people who are incapable/unwilling/don't make a serious enough effort to *both* pursue interesting tangents that could later be developed into other full-length conversations *and* stay on topic overall. Granted, this probably describes a vast majority of people. Nevertheless, presuming the straying from topic though tangent acknowledgment to be an affliction of conversation with *all* people unfortunately leads you imply a necessary trade-off between the values of rigorous word definitions and untangling all those "really important" topics. While I come to this from a weakness for distinction-making, I don't think that weakness really impairs my resolve to get through what you imply (and what I'm strongly inclined to agree) are the larger topics. In other words, I haven't found interests in semantic and non-semantic questions to be mutually exclusive.

While I agree that "Redefining a word won't change the facts of history one way or the other," I find the "this is exactly the wrong way to look at the problem. What you really want to know - what the argument was originally about" part of the complaint to be both unpleasantly constraining and inaccurate. That there isn't "exactly the wrong way to look at a problem" -- what it *is* is defining a whole other area of interest. If in fact every time such a new area of interest is defined some other earlier problem risks getting abandoned altogether, then sure, I agree, that's absolutely no good, but I just haven't found that to be the necessary case with all conversations.

With this comment I'll only express a tiny insight into the semantic part of the conversation (and I understand that addressing it doesn't actually get at what you hope to discuss at full length, but whatever.) The whole question of whether "someone who has a definite opinion about the existence of at least one God, e.g., assigning a probability lower than 10% or higher than 90% to the existence of Zeus" should be called a "religious" person can be niftily neutralized by a slight but, I think, helpful rephrasing of same: Should that person be considered to hold religious opinions? If you agree that this new question doesn't omit anything interesting from the original question (and you may not), then you may notice that the added benefit of such a rephrasing is that it blocks that whole silly digression about Stalin's religion being Communism.

If you're off-put by this kind of nitpickiness, perhaps you should reconsider: I think that getting your interlocutor to recognize that he or she is introducing an entirely new topic -- semantics -- rather than expanding on an original one may help you both remember that an answer to the semantic question doesn't even begin to address the non-semantic question. (This isn't true in all cases, but it is in this one.)

I find that kind of distinction-making valuable because it doesn't limit the the topics "worthy" of consideration *and* ensures that interesting questions don't get abandoned. Putting aside the opportunity cost of one discussion over the other, everyone should be happy about this, no?

Comment author: Rolf_Nelson2 09 December 2007 10:39:00PM 0 points [-]

Rolf: It seems to me that you are trying to assert that it is normative for agents to behave in a certain manner *because the agents you are addressing are presumably non-normative*.

On a semantic level, I agree; I actually avoided using the word "normative" in my comment because you had, earlier, correctly criticized my use of the word on my blog.

I try to consistently consider myself as part of an ensemble of flawed humans. (It's not easy, and I often fail.) To be more rigorous, I would want to condition my reasoning on the fact that I'm one of the flawed humans who attempts to adjust for the fact that he himself is a flawed human. (But, I don't think that, in practice, this particular conditioning would change my conclusions.)

I do have to, to "bootstrap" my philosophy, presume that I have some ability to, much of the time, use logic, in such a way that (on average) >50% of my 1-bit beliefs are likely to be correct. But since I grant that same ability to the rest of the ensemble of flawed humans, that doesn't affect the analysis.

I don't have a citation to an existing paper that rigorously spells out how you would do this (maybe such a paper doesn't even exist, for all I know), but my intuition is that such analysis is not, at a fundamental level, self-contradictory.

Comment author: [deleted] 10 December 2007 12:07:00AM 0 points [-]

About the word "supernatural"... isn't this whole concept not bogus?

This is obvious if we assume the there was No Design(er). However the same applies more widely. Anything that happens or exists will be natural by definition.

Consider this: Let's assume for arguments sake that Intelligent Design is correct. Then all of nature is created and hence unnatural by definition. Depending on the nature of the designer it might be the only really natural thing around.

Conclusion: So unless intelligence is considered supernatural, the simple conclusion must be that nothing supernatural exists.

Comment author: JB7 20 December 2007 04:05:00PM 0 points [-]

This has of course been pointed out, but "And it is triple ultra forbidden to respond to criticism with violence... Does not get bullet. Never. Never ever never for ever." --- doesn't make sense. It's a bizarre authoritative / normative assertion completely out of place in all of this. And Eli knows that, I'm sure. I suspect that he's just had this directive beat into him in previous encounters, in objection to the "coldness" of purely utilitarian decision-making. If e.g. an argument-ending bullet is what it takes to ensure humanity gets off-planet, then the utilitarian answer is to shoot.

Of course.

Comment author: Doug_S. 14 February 2008 07:21:00PM 1 point [-]

There is one type of claim for which a bullet does provide relevant evidence: claims about bullets themselves, such as "Your bullets will not harm me." (For example, an armored vehicle that is supposed to protect its occupants from gunfire will certainly end up being tested against actual gunfire at some point before its design is put into widespread use.)

Comment author: Robin_Z 18 May 2008 02:24:00PM 0 points [-]

Not replying to the comment thread: I think the quote might actually be Deuteronomy 13:6-10 in the King James Version.

Comment author: idlewire 15 July 2009 07:24:26PM 1 point [-]

(Deuteronomy 13:7-11)

Talk about a successful meme strategy! No wonder we still have this religion today. It killed off its competitors.

Comment author: kpreid 13 September 2010 07:45:56PM 1 point [-]

Crosslinking:

If you don't place an appropriate burden of proof on each and every additional nice claim, the affective resonance gets started very easily. Look at the poor New Agers. Christianity developed defenses against criticism, arguing for the wonders of faith; New Agers culturally inherit the cached thought that faith is positive, but lack Christianity's exclusionary scripture to keep out competing memes. New Agers end up in happy death spirals around stars, trees, magnets, diets, spells, unicorns...

In the August 2010 open thread, Risto_Saarelma linked to this relevant article from an insider’s perspective on (if I recall correctly) something like the above matter:

Bridging the Chasm between Two Cultures: A former New Age author writes about slowly coming to realize New Age is mostly bunk and that the skeptic community actually might have a good idea about keeping people from messing themselves up. Also about how hard it is to open a genuine dialogue with the New Age culture, which has set up pretty formidable defenses to perpetuate itself.

Comment author: BlindDancer 03 April 2011 01:42:21PM 0 points [-]

Look I know this is a stupid quibble. But I have a quibble. One of those little things that just bugs me and snaps me out of reading the article specifically to complain about it.

Stalin was religious and lifted the ban on religions put in place by Lenin. Stalin trained in being a minister before he started being a communist. (Check wikipedia for evidence)

That having been said: Other then that this article is pretty good, and I can understand not wanting to actually get into the religion vs aethisim debate or spending the time to correct my quibble... Other then, you know, the fact that you're saying that the fact that I require burden of proof there is a good thing.

The problem is that every core assumption you take requires a certain amount of faith. (See the mock turtle). Most core assumptions are based on emotions (which are humanity's way of weighing the value of core assumptions, if used properly) and emotions are based of a mess of learned responses and instincts that operate at a level that is difficult to control intellectually. (Instinct: I have a certain type of pain, therefore I am hungry. Therefore the value of food raises in my mind)

Which is why, of course, every rational debate comes down to differences in base assumptions which are, of course, difficult to change, because base assumptions are rooted in emotions, cultural heritage, and instinct.

Comment author: TheOtherDave 03 April 2011 03:46:27PM 2 points [-]

every rational debate comes down to differences in base assumptions which are, of course, difficult to change, because base assumptions are rooted in emotions, cultural heritage, and instinct.

Sometimes, I have debates with people where it turns out that our disagreement is not due to differences in our base assumptions rooted in emotions, cultural heritage, and instinct, but rather due to differences in our proximal observations of the world, or our recollections, or our reasoning processes, or various other things.

I had thought this was true of everyone... is it not the case for you?

Comment author: AGirlAlone 10 February 2012 10:37:17AM 0 points [-]

Concerning many comments already here that I am not sure which one I should reply to:

Never an argument to warrant violence? Or OK against superintelligences but NO against humans? Do not suppose there's a sharp line between human and superintelligence situations. To me some of you may well be akin to superintelligences, that I cannot outwit. No absolute line between argument and verbal abuse either, when I think about it. Also, I think I have some examples of dangerous/disgusting arguments - nothing exists, you should die, your consciousness doesn't exist ...

As for whether the rightness of a violent arguments has to do with the physical power of the opponent -

Should you let the moral value of initiating violence depend on whether or not you win?

I say yes, but my idea of moral value is more self-centered. My morals consider others, but I think it's moral to prefer to survive - not the least because if your moral doesn't prescribe survival, you will not be here. It's not as if we help others out of morals and survive out of baser urges. That dichotomy is common in present morals ( think bioethics - if you don't accept death, you refuse to "open up to higher goals"/live for others) but it's nonetheless sick. It's right and moral to want to survive! And thus I decide that while arguments should be free when you are only concerned with truth and rationality, in the case of lots of real situations, it's more than truth at stake, and you worry for your well-being. Even if you want to keep it at the rational, intellectual level, your opponent may not oblige. And then it would be moral to use violence, but not moral to risk your own life for small arguments, but not because of the value of truth or laws of rationality at all.

Though even then I wish to be more intelligent beforehand in preparation for such a sad event, so that I may be strong and integral enough to know the offending argument without being hurt, and do not have to use violence, or at least ponder their point after the violence safely.

Comment author: Rixie 29 October 2012 03:23:07AM -3 points [-]

If God is real, and he actually said the thing about killing people who want to divert you from your faith, then he has every right because he is the one true god etc., and if God doesn't exist and he didn't say that, then Christianity IS a bad thing.

Comment author: chaosmosis 29 October 2012 03:35:40AM 4 points [-]

Possibly, but only if you think that morality would stem from authority even if God existed. Some people would disagree and say that humans should do what they think is right even if there is a God who tells them to do otherwise.

Comment author: Rixie 01 November 2012 11:54:42PM -3 points [-]

But if God made people and our entire existence is up to his design, then a) He know's something we don't or b) The world is a cruel, cruel place.

And anyway, if he created humans, he planted our sense of morality too. I think the lesson here is: Don't go chasing after other gods or you will get killed.

Comment author: chaosmosis 02 November 2012 12:43:41AM 2 points [-]

I don't understand how any of those three possibilities refute what I said.

Comment author: Abd 02 November 2012 02:27:13AM -1 points [-]

They don't. However, what you said posed a created contradiction. There is more than that, to be sure. Rixie is saying one piece of this: implying that there will be no contradiction, because your "sense of morality" comes from God. I think that's a bit naive, but not totally off. That is, I can think all kinds of crazy stuff. That's not the same as knowledge.

The "other gods" implies a context where there is one, yours. What is that? Is this your identity or is it something deeper? If Rixie is right, "chasing" some other source of meaning could be fatal. What could that mean?

Chaosmosis, you objected elsewhere to my capitalizing Reality, I think. Reality is my substitute-name for God. So if I am chasing another "god" besides Reality, I'm literally going crazy.

Comment author: chaosmosis 02 November 2012 03:35:29AM -1 points [-]

Rixie's point was operating within the hypothetical of "If God is real, and he actually said the thing about killing people". Rixie defended that point with an appeal to authority. I pointed out that appealing to God's authority only works if we believe what God said. That somewhat invalidated Rixie's initial point. If Rixie now wants to argue that God would never order us to do something wrong, or something else like that, then Rixie needs to not only point out that the claim isn't necessarily founded on an appeal to authority but also needs to add an additional argument which the claim could be legitimately founded on. I initially didn't catch on that Rixie was abandoning the hypothetical because Rixie never took that second step.

I don't understand your middle paragraph. What context is this in, where is the "other gods" quote from?

I can't understand your last paragraph very well until I understand the relevancy of the middle one. I objected to capitalizing words like Reality because that sort of capitalization only occurs when you think of something like a proper noun, and thinking of things like proper nouns when the things aren't proper nouns often adds a level of mysticism to the concept of those things which is detrimental to rationality.

Comment author: Abd 02 November 2012 04:09:52AM *  -2 points [-]

Okay. First, I don't really care who said what when, my goal is something like consensus or shared vision, and how we get there, and who stumbled and how, isn't so important to me.

"other gods" was from Rixie:

And anyway, if he created humans, he planted our sense of morality too. I think the lesson here is: Don't go chasing after other gods or you will get killed.

This is proposed as a message to you. (You could choose to take it impersonally, as having nothing to do with you, but Rixie did use "you." That might be accidental, but I like to start with what people actually wrote or said. Kind of the point I've been making here, in fact.)

So someone says to you, chaosmosis, "don't go chasing after other gods, or you will get killed." The principle of exegesis that I was taught was to assume that statements are correct. In fact, every statement can be true or false, but assuming that statements from others are true is the most powerful place to start. Assuming that the truth is literal and fixed would be irrational; this is just about communication process. When people start with skepticism and rejection, they can only come to understand statements when they are lucky enough to find serious proof. That's actually rare, the approach is highly inefficient.

So what would "other gods" mean for you? I proposed a meaning. What do you think of it?

"Other god" implies "God." What would your God be if the words meant something? Rather than anticipate an answer, I'll stop.

As to capitalization, I use it with proper nouns. You assume that words are proper nouns or not. That depends on context. What you think of as "mysticism" could be a level of meaning that perhaps you don't recognize, and you assume it is detrimental to rationality. Is that a fact? How would you know? What, indeed, is "mysticism"?

Comment author: chaosmosis 02 November 2012 05:53:11PM *  -1 points [-]

The principle of exegesis that I was taught was to assume that statements are correct. In fact, every statement can be true or false, but assuming that statements from others are true is the most powerful place to start. Assuming that the truth is literal and fixed would be irrational; this is just about communication process. When people start with skepticism and rejection, they can only come to understand statements when they are lucky enough to find serious proof. That's actually rare, the approach is highly inefficient.

You contend that starting by believing what others say to be true is true is the fastest way to truth. I disagree. I think we should take others opinions as evidence, but that we should evaluate truth on a probabilistic level. There are no defaults, and we shouldn't unfairly privilege any hypotheses. I think the truths you arrive at through that method aren't true if they can't be arrived at through the other method. People say many false and nonfalsifiable things. People assert things that they have no way of knowing or that make no sense at all. There is no reason I should believe people in these cases.

You conflate different instances of claims by people, essentially, viewing them all as equal. I make distinctions, and say that people's opinions are good approximators of the truth in some cases but bad in other cases. This seems faster because it ensures that I don't get stuck whenever I hear someone make a nonfalsifiable claim. There is an invisible and untouchable dragon behind you who will eat you and send you to hell if you believe nonfalsifiable claims. If you truly followed the system where you believe everything anyone tells you until you get contrary evidence, you would now have a terrible paradox on your hands.

How do you make the jump from this communications based model of evidence to a model which incorporates evidence? It seems like there's a huge disconnect there, if the default mode is acceptance of others ideas then there would never be any reason to make the jump towards evidence.

So what would "other gods" mean for you? I proposed a meaning. What do you think of it?

"Other god" implies "God." What would your God be if the words meant something? Rather than anticipate an answer, I'll stop.

I don't think I have any god, so I don't know what "other gods" might mean for me. I'll speak metaphorically then. I would say that to the extent that I have a god, it's a nonomniscient and nonomnipotent god that I find sort of pathetic most of the time, and his name is chaosmosis. Chasing other gods would be impossible for me because everywhere I go there I am. However, I can change as an individual, and I can pursue "other gods" by changing myself.

Thinking like this is relaxing and entertaining but not useful. I don't mistake this for truth. It might be true, but the process that brought me there was a lazy and invalid one, and in other cases it would fail. It would be right for the wrong reasons.

As to capitalization, I use it with proper nouns. You assume that words are proper nouns or not. That depends on context. What you think of as "mysticism" could be a level of meaning that perhaps you don't recognize, and you assume it is detrimental to rationality. Is that a fact? How would you know? What, indeed, is "mysticism"?

I think that proper nouns should only be used where they are used traditionally.

Mysticism is an emotion of awe and humility and grandeur. Emotions are not evidence. I'm human, so the temptation is for me to treat emotions as evidence. This is detrimental to rationality. Therefore, I try to avoid my encounters with seductive emotions like mysticism, or to accept those encounters but also to make sure that I'm justifying my decisions on a logical basis and not an emotional one.

Mysticism is meaningful, but in a subjective and emotional sense. In a logical sense, it fights against meaning and truth.

Comment author: Abd 02 November 2012 02:17:46AM -3 points [-]

"Some people" (i.e., me) would say that God tells you to do what you know is right. (In Arabic, ma'ruwf, "good," the root is 'arifa, implying knowledge.) So, on the authority of God, do so, even if someone tries to tell you God wants you to do something else. Today you don't need to kill him, but he might be trying to kill you, i.e., to destroy your freedom.

If God tells you, directly, to do something else, try getting the right medication, you've got some killer voice in there. God doesn't do that.

If your name is Abraham, my condolences.

I hope people don't mind my tossing in the perspective of a Muslim rationalist here. Not all Muslims think this way; in fact, the particular school is the Mu'tazila, though the errors and maybe some innovations are mine.

Comment author: chaosmosis 02 November 2012 03:52:12AM *  -1 points [-]

If God only tells you to do what you know is right, that's fine. God's authority doesn't add anything to the obligation though. It would be just as right without a God as there is with it.

I have problems with the idea that God is the source of our understanding of right and wrong though. If you accept that idea then contradictory preferences are evidence against the existence of God, and it's pretty clear to me that preferences contradict all the time. Plus, I don't think you're justified in ignoring the Abraham data point, it seems like a significant and revealing event.

Comment author: Abd 02 November 2012 04:57:59AM *  -3 points [-]

It would be just as right without a God as ... with it.

Agreed. However, do you ever need to be reminded of something?

the idea that God is the source of our understanding of right and wrong...

A bit of a fish bicycle, eh? Or, a tautology? At best, the idea is not a truth, it is a way of looking at life. It either works or it doesn't. It's empowering or not. That's all. If used to make a personal sense of right and wrong into an absolute, it's been corrupted. Contradiction of preferences is decent evidence that the personal sense isn't absolute, but that doesn't bear on the existence of God. God made us different, that we might recognize each other. That, again, is Qur'an.

Most arguments for and against the existence of God are rather equally silly. I used to lecture university students on Islam, and once an atheist proudly asserted his position, "I don't believe in God." I asked him, "What God don't you believe in?" I think he was already a bit confused, I forget his answer, if there was one. I then said, "The God that you don't believe in, I don't believe in either." He was speechless. He'd expected an argument. Of course, it was an argument, just one he'd never heard before. I wasn't being silly, and I was telling him what was true for me.

(I was more combative in those days. Now, I'm a little ashamed to tell that story. Did I actually communicate something to him -- or discover something with him --, or did I merely humiliate him? The fact that I don't know indicates something was missing.)

I didn't "ignore the Abraham data point." After all, I brought it up. To cover the Aqedah, though, could take more than I'm prepared to address today. I wrote, "my condolences," because that was truly a trial. Can you imagine?

I have seven children. Abraham didn't have the option to check himself into a psychiatric unit. I would. Still, somehow it worked out anyway.

And, should I mention that it is just a story? Again, a Muslim view, from the Qur'an: God tells stories.

There are more details in the Qur'anic story, but it's not really necessary here so I'll leave it.

Comment author: chaosmosis 02 November 2012 05:31:17PM -1 points [-]

A bit of a fish bicycle, eh? Or, a tautology? At best, the idea is not a truth, it is a way of looking at life. It either works or it doesn't. It's empowering or not. That's all. If used to make a personal sense of right and wrong into an absolute, it's been corrupted. Contradiction of preferences is decent evidence that the personal sense isn't absolute, but that doesn't bear on the existence of God. God made us different, that we might recognize each other. That, again, is Qur'an.

I wasn't talking about contradictory preferences between different people, but within one individual person.

I want to be famous but also want to not talk to people. I want to be strong and a hard worker and smart and also want to sit on the couch all day watching TV. I want to be happy but I also enjoy moments of extreme sadness. My preferences don't make sense at all, they're not coherent and they change over time and they're shaped significantly by my genetics and my childhood environment, so it's extremely more probable that they're the product of a random process like evolution than that they're the product of a God. If a God is responsible for my preferences, he is insane and incompetent.

Most arguments for and against the existence of God are rather equally silly. I used to lecture university students on Islam, and once an atheist proudly asserted his position, "I don't believe in God." I asked him, "What God don't you believe in?" I think he was already a bit confused, I forget his answer, if there was one. I then said, "The God that you don't believe in, I don't believe in either." He was speechless. He'd expected an argument. Of course, it was an argument, just one he'd never heard before. I wasn't being silly, and I was telling him what was true for me.

Your ability to confuse one naive college atheist isn't very strong evidence for anything. The atheist should have replied that he didn't believe in any gods. You weren't making an argument, just confusing him. Or if you were making an argument, then I've missed it, and I need the premises more clearly stated.

(I was more combative in those days. Now, I'm a little ashamed to tell that story. Did I actually communicate something to him -- or discover something with him --, or did I merely humiliate him? The fact that I don't know indicates something was missing.)

I think you probably confused him, or he was bad at thinking on his feet and responding to new ideas, but that afterwards he thought about it and figured out what his reply should have been. I would expect that "I don't believe in any gods" is a fairly obvious reply to most people. He was probably annoyed with himself, and abit upset at you for confusing him, and he might have been a little embarrassed too.

I didn't "ignore the Abraham data point." After all, I brought it up. To cover the Aqedah, though, could take more than I'm prepared to address today. I wrote, "my condolences," because that was truly a trial. Can you imagine?

That's fine, it's a big issue. I felt like you were lampshading it, but if you've got a lengthy complicated explanation of it that you'd rather not go into I sympathize and agree. I'd rather not go into detail with it either.

Comment author: Abd 02 November 2012 09:24:25PM *  -3 points [-]

If a God is responsible for my preferences, he is insane and incompetent.

Great. What is this thing called "responsible"?

There are at least three major possibilities here, once we get past a definition. Feel free to add other possibilities.

  • God is responsible.

  • You are responsible.

  • Nobody is responsible, these are just circumstances, meaningless.

The conclusion (insane and incompetent) does not follow from the premise, however. It requires unspecified assumptions. Which one of these shall we examine first?

Comment author: chaosmosis 03 November 2012 03:11:50AM 0 points [-]

I don't mean responsible, I guess. That was poor phrasing.

If God intentionally chose my preferences and my preferences contradict in ways that don't make sense then God is crazy or incompetent or doesn't care about my preferences because my preferences wouldn't contradict if they were designed for any sort of reason.

Comment author: Abd 03 November 2012 03:29:51AM -2 points [-]

In most of these conversations, God is a fish bicycle, isn't that obvious? There is a possible conversation about what "God chose," but the theologies that make sense to me essentially zero out the God contribution, beyond creating some expectation that it all makes, in the end, some kind of sense, and that we can search for that. God is, I'll just say, outside of time and so placing God's choice in the past, again, makes no sense. What is "God," anyway?

Since you don't accept the idea of God, whatever reasoning you create about God is made up, fantasy about fantasy.

Was there any choice at all involved in the creation of your "contradictory preferences"? What are you talking about? What is "choice"?

Okay, the core: you assume that contradictory preferences mean that something is wrong. Otherwise you would not conclude from them that God is crazy or incompetent. I'll just make up something not-wrong. You have differing preferences because they give you different points of view, and when you can see from more than one point of view, you get depth perception, right? Is depth perception valuable?

My own answer, by the way, about the three possibilities is All of the Above. Those are simply three stories we can tell about responsibility. Each one produces, if held in mind, consequences. Each has a value, but generally the most empowering is the second: you are responsible for your identity, which includes what you describe as your preferences.

Isn't that obvious? Okay, maybe it's not. We don't ordinarily think of ourselves as something that we created, we tend to "blame" it on our parents, society, or circumstances. But our identity was formed out of how we reacted to those factors. Did we chose these reactions or were they just automatic and predetermined? I'll leave the question there for now.

Comment author: shminux 02 November 2012 10:08:53PM 4 points [-]

I hope people don't mind my tossing in the perspective of a Muslim rationalist here.

I am somewhat curious how you define "Muslim rationalist" and how such a rationalist is different from, say, a Sikh rationalist, and whether the two can come to an Aumann agreement in religious matters.

Comment author: Abd 02 November 2012 10:23:32PM *  -2 points [-]

You may be interested in difference, I'm more interested in agreement. I gave a link to the Wikipedia article on the Mu'tazila, not because I'm an "adherent" of the school, but simply to connect myself with Muslim tradition. I would expect, however, that a rationalist will find ample agreement with any other rationalist, as long as they have some substantial shared experience.

Thanks for the link on Aumann agreement. From the article, I find this hilarious.

Aumann's agreement theorem says that two people acting rationally (in a certain precise sense) and with common knowledge of each other's beliefs cannot agree to disagree. More specifically, if two people are genuine Bayesian rationalists with common priors, and if they each have common knowledge of their individual posteriors, then their posteriors must be equal.

I know that's not what is meant, but the obvious pun is beautiful. If I know my own ass (presumably from a hole in the ground), I will know the ass of others who likewise know their own ass.

However, I do think that we can agree to disagree. Not necessarily rationally, that's all, as to rational completeness. Sometimes we make choices that the work involved in tracking down the necessary conflicting assumptions or reasoning is not worth the expected gain.

The assumption of Aumann's theorem is an empowering one, because it leads to expectation of agreement, making it far easier to find -- and build or create -- than with a contrary expectation.

Comment author: shminux 02 November 2012 10:29:14PM *  3 points [-]

You may be interested in difference

No, I am interested in the definition, and you haven't given one. My guess is that a qualifier for rationalism indicates a certain degree of compartmentalization, a refusal to give up some cherished beliefs. So I expect these beliefs to show up in your definition of a Muslim rationalist.

And yeah, the unintentional pun is hilarious, but also instructive: "I know my own ass" tends to be a false statement, as people are notoriously bad at noticing "the log in one's own eye", sorry for the biblical reference.

Comment author: Abd 03 November 2012 03:06:09AM *  0 points [-]

You asked about the difference. "Muslim" was not a "qualifier" of "rationalist." I'm a rationalist who is a Muslim. I could just have easily have written "rationalist Muslim." Your expectation was not unreasonable, but a poor guess.

I was using "ass" to refer to knowledge of self, including one's one "assholery." It's true that most people don't know it -- or won't admit it.

Comment author: shminux 03 November 2012 07:17:50AM 4 points [-]

I'm a rationalist who is a Muslim.

I'm just having trouble reconciling faith with rationality, so I hoped you'd explain how you do it.

Comment author: Lu93 02 June 2014 11:05:14PM 0 points [-]

While reading, i tried to think of a case when i fell in affective death spiral, and interesting thing came to my mind. Falling in love falls under Halo Effect? Butterflies in stomach, worshiping the beloved, etc... That means that who overcomes this bias can't fall in love that way anymore?

Comment author: Viliam_Bur 06 August 2014 02:55:56PM 1 point [-]

There is a difference between infatuation and love. (Similar to the difference between "Hollywood rationality" and rationality.) Affective death spiral is infatuation. A person who overcomes this bias will not say things like: "Oh, if this amazing person I met five minutes ago will not friend me on facebook then my life has no meaning and I have to slash my wrists."

Comment author: Lu93 01 September 2014 11:22:51AM 0 points [-]

Yes, infatuation is what i really wanted to say.(I'm not native speaker) So, two points: 1. Affective death spiral has leading role in existence of humanity, (if none had it, less children would be born.) 2. It's kinda shitty to find out that butterflies are consequence of false beliefs, which could lead to people being resistant to accepting this whole idea.

Comment author: emhs 06 August 2014 08:53:46AM 0 points [-]

Am I the only one who hears Eliezer's "Never ever never for ever" voiced roughly like HJPEV?