When did Tyler say that overcoming bias is not important? Are you talking about this post here: http://www.marginalrevolution.com/marginalrevolution/2007/08/how-important-i.html

He says it's not the most important thing in the world, not that it's not important. So, is your reading of Cowen biased? Or did I miss something?

J, that's Tyler quoting a summary of Hanson, not Tyler himself.

Robin, I think it'd be pretty silly to trade off social graces against the Inquisition, especially when some creative thinking could devise alternative social graces.

Seriously, we're trading off social graces against what?

In all human history, every great leap forward has been driven by a new clarity of thought. Except for a few natural catastrophes, every great woe has been driven by a stupidity. Our last enemy is ourselves; and this is a war, and we are soldiers.

Amazing quote.

Summary: if they can make you believe absurdities, they can make you commit atrocities.

Amen.

"In my opinion, this is Establishment writing (Mao and Stalin as the "bad guys") only problematic to me because the strength against challenges of your presented ideas nontransparently rest on power alignment, rather than just their accuracy of modeling apparent reality and devising solutions to challenges we face."

Don't you love post modernists? their rhetoric is frankly baffling to common sense (what they would term a mental power-structure - I'm not sure even they understand what it is their writing half the time)

Its obviously been long since humanity had to strive for objectivity and earnest disinterest because human bias was creating so much suffering in the world.

There isn't much known about Hopefully Anonymous, but I'm fairly certain he does not consider himself a post-modernist.

mtraven - To be against bias is not the same as to be without bias, so even if we were to grant that James is "comfortably against bias and for torture without losing a moment's sleep", that would not constitute a counterexample to Eliezer's claim that "where there is...torture...there are biases enshrouding it."

Flynn, response on my blog within the next ten minutes ("being different" isn't what I mean).

http://www.hopeanon.typepad.com

Flynn, HA is a near-total egoist who wants to live forever.

Why would a near-total egoist hope to be anonymous? :-)

HA, replied on your blog. Sorry about the confusion.

Flynn,

Egoist!=egotist.

There's a huge positive chunk of hypocrisy that we're missing there - hypocrisy allows us to have ideals higher than we can (and do) attain in our actions. It can have a tremendously aspirational effect. The phrase "all men are created equal" was written by rich, white slaveowners. Eliezer feels that if hypocrisy had been banned, they would have written the same phrase, and set all their slaves free. I fear that if hypocrisy had been banned, they would have kept their slaves and instead written "all rich, white men are created equal". And future progress would have been ruled out.

Ah, now there's a powerful argument.

But at the same time, if hypocrisy had not decreased, we would still have rich white slaveowners.

It should be assumed by default that when I talk about the benefits of overcoming bias, I am talking about the sort of human being who comes into existence when they set out to overcome their own biases by acts of mental will and training. Not, necessarily, the sort of entity that you get if you do neurosurgery on a human; nor the sort of entity that would have evolved if deception and self-deception had not been part of the ancestral environment.

The sort... (read more)

Eliezer, you seem to say that trying to reduce hypocrisy is good for people who are trying to overcome their biases. That seems a tautology, and doesn't seem that related to the issue you seemed to raise in the post, which is whether hypocrisy in society tends to promote holocausts.

"Helping people to open their eyes and see human suffering, raising children to be compassionate, will do far more to get rid of the Hitlers and Castros than logic and writing classes."

'Cause no one ever thought that Che Guivera or Lenin were acting out of a compassionate desire to end suffering. Nobody EVER claimed that.

Confucius on the Rectification of Names written some 2500 years ago.

"Tzu-lu asked 'If the Lord of Wei entrusts the government to you, what will you do first?"

"Correct names, surely!" the Master said.

"How can you stray so far from the point! What would that correct?"

"Tzu-lu, you are a boor. On matters of which he is ignorant a gentleman expresses no opinion. If names are incorrect, it is impossible to speak. When it is impossible to speak, work is not done. When work is not done, society breaks down and punishment is ... (read more)

David Brin has a nice analysis in his book The Transparent Society of what makes open societies work so well (no doubt distilled from others). Essentially it is the freedom to criticize and hold accountable that keeps powerful institutions honest and effective. While most people do not care or dare enough there are enough "antibodies" in a healthy open society to maintain it, even when the "antibodies" themselves may not always be entirely sane (there is a kind of social "peer review" going on here among the criticisms).

Muddle... (read more)

In my post here you can find links to defenses of Stalin and Mao. They do not deny that both killed a huge amount of people, or that a great many of those people were completely innocent and it was tragic that they were killed. The author instead states that Stalin and Mao gave enough benefits to people, who had previously been faced with even worse rule, that those unfortunate deaths should be viewed as cons outweighed by the pros. I can't do an effective job of presenting his case (for one thing I completely disagree with it and think the two were among ... (read more)

While Eliezer and I may be approaching the topic differently, I think we have very much the same aim. My approach will however never produce anything worthy to go into anybody's quote file.

Anders & TGGP, Response on my blog within 10 minutes (so as not to flood overcomingbias with posts).

How come this blog won't remember my info? I keep clicking that damn checkbox to no effect.

I read Orwell's essay last night. Quite impressive, but I didn't immediately understand some of his criticisms. I did later on, and would like to share.

One thing that I had a revelation about is this pair of equivalent passages:

"I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to th... (read more)

I think the recent 1984 quotes are close to encouraging generalization from fictional evidence. Bias and totalitarianism are not more connected because Orwell wrote a work of fiction connecting them; you may understand this on a gut level, but many readers probably don't. If there's a duty to make history available, isn't there a converse duty to make fiction unavailable?

Anyway, here are my thoughts on whether it's a good idea to choose bias.

The rules of the game are much different when you have bad faith, as in totalitarian societies, as opposed to comparing Democrats vs Republicans. Or are they?

steven, your criticism is valid only if Eliezer used the passage from 1984 in some way that properly requires that it actually have happened. It is not at all clear that he did so. For example, if he used it as a way to illustrate a point, clarifying his meaning, then it was not improperly used. If he used it as evidence of Orwell's own well-thought-out views on the subject, then again, it was not improperly used. There are many ways to use fiction that are not improper.

"For example, if he used it as a way to illustrate a point, clarifying his meaning, then it was not improperly used. If he used it as evidence of Orwell's own well-thought-out views on the subject, then again, it was not improperly used. There are many ways to use fiction that are not improper."

What his intentions were doesn't matter; what matters is the expected reaction of the audience, which in this case is going to see a vivid example supporting Eliezer's claim that human evil sets out to muddle thinking. This happens to be true, but it doesn... (read more)

Copy of my post defending Stalin and Mao from Hopeanon.

Hello, my defense of Stalin and Mao are simple and the links can be found here on the Entitled To an Opinion blog. My argument is simple. Stalin saved far more lives than he took. In fact, Czarism was three times more deadly on a per capita basis than the average for the Stalin years. Plus, Stalin set a world record for the fastest doubling of life expectancy in any land. This amazing feat was only broken by Mao in 1976. Therefore, based on those records, I hold that Stalin and Mao were two of the grea... (read more)

This is a pretty interesting discussion. While the overall topic of this blog is the worthwhileness of overcoming bias, I think how that relates to "evil" is a pretty important facet of what we should talk about. Some of the comments on this post reminded me of a passage from a novel (a fantasy novel if you must know) on the nature of evil. I found it to be very profound and I think everyone here might find it sort of interesting. You'll have to excuse the use of some of the plot specific names, the beginning of each chapter of the book opens wit... (read more)

mtraven, You're ignoring concepts like inquiring empirically whether situationally employed torture can reduce net torture in the world. I brought it up in the torture thread and I think every commenter ignored the concept. Also, I'm unconvinced that the desire to label indivuduals as "good" and "evil" comes from a good faith attempt to accurately model reality, or even optimally solve existential challenges we face. It seems to come more from a desire to use morality to create status heirarchies, although in some cases it could also cr... (read more)

HA, that's exactly the sort of argument I'm talking about. It is too easy to convince oneself by some bit of reasoning that doing an evil act is OK -- maybe it reduces some other evil, maybe it gets rid of the Jews who you have convinced yourself are a source of evil. Maybe you can convince yourself that implementing some torture will reduce the total amount of suffering in the world. I would be extremely dubious. Reasoning from first principles about practical affairs is extremely unreliable, and has to be augmented with heuristics, intuition, and gut ... (read more)

Tiiba, I've always considered this bible quote

"I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all."

to be a paradigm of bad writing. The nasty trick it exemplifies is using a paradox to get one up on the reader without committing to a specific meaning.

If the race is not to the swift, who does win? The lucky? Contrast two aphorisms "the race is to the sw... (read more)

"Pro-torture and anti-torture factions will have to fight it out politically, just as the pro- and anti-homosexuality factions do. Which side are you on?"

haha, that reads as deliberate bait to me. Presenting 2 options as if they're the only options seems to be preying on a dualistic/binary bias common in most people, and that I think stems from our primate roots. That sentence reads like it could be lifted from a George W. Bush speech, that type that's widely mocked for performing a lack of nuance.

When we think in ways that hide the consequences of our actions, there's no way we can make good choices based on consequences.

But when that isn't blurred we still might disagree about the choices. So if you don't think DDT affects songbirds you're likely to think DDT is entirely a good thing. But even when you know, you might decide that perfect fruits are more important than birds.

Moral choices aren't inevitable when you understand the facts, but without the facts it's even harder.

There's often a conflict between immediate things and distant implication.... (read more)

J Thomas, I really like your last paragraph "Still, sometimes the language is central. When we say the US military is in iraq "protecting iraqis", it might work better to reduce the level of abstraction and say we're there "killing people and blowing things up". Or maybe we're "killing people that we think are about to kill the ones we don't want killed". When we say "train the iraqi army" we might say "train selected iraqis to kill people and blow things up under our direction". The clearer we say what we're doing, the easier it might get to create a better strategy."

I'm sure TGGP will agree with my request to you: Start a blog!

that reads as deliberate bait to me. Presenting 2 options as if they're the only options seems to be preying on a dualistic/binary bias common in most people, and that I think stems from our primate roots.

Everything we do stems from our primate roots. I don't know you well enough to deliberately bait you, but using that expression (which comes from ) is meant to signify that at some point, when dealing with politics, you have to stop spouting hot air, choose which side of an issue you are on, and fight for it. That attitude seems somewhat antithetical to ... (read more)

Whoops, messed up a link there, was supposed to read: comes from an old union song. The link is to my own blog where I am basically taking the opposing view, that forced-side-choosing is bad, in many circumstances. But I did say I was contrarian...

mtraven, my position is pretty transparent. I want to persist as a subjective conscious entity, forever, and everything else is subsidiary to that. To the degree torture (of myself or others) maximizes my persistence odds, then I perceive it to be in my interest to welcome it. To the degree torture (of myself or others) reduces my persistence odds, I oppose it.

To the degree you mtraven would place opposing torture over your own persistence as a subjective conscious entity is perhaps a triumph of meme over a subjective conscious individual, in my opinion.

Al... (read more)

mtraven, I'm not sure what it means to be "objective, apolitical" but I certainly hope that when issues arise here people here are willing to set aside what they think they know enough to imagine being uncertain and then trying to evaluate which way the evidence and analysis leans.

I'm puzzled, as usual, but perhaps more so: this post has helped clarify the lack of clarity in my understanding of "bias" as the word is used here. You see, I don't in general see an a priori distinction between "bias" and other kinds of heuristics; we are talking about computational shortcuts, ways to save on reasoning, and all of them go wrong sometimes. I'm glad to have scope insensitivity pointed out to me, and having seen the discussion on this blog may even keep me from some error at some time, but my reasoning will always be inc... (read more)

HA, you may have your personal single life goal worked out but I'm not sure why the rest of us should be interested. My goal is emphatically not persisting my subjective reality as far forward in time as possible, let alone yours. I have other people I care about, I probably care more about the quality of my life than quantity, I do have memes that I place a great deal of value on. Most other normal people feel the same way, I suspect.,

Robin, what evidence has been brought forth to support the proposition that torture may be a good idea? I have cited boo... (read more)

"Carl Sagan (I think) said we should be open-minded, but not so open that our brains fall out. It's even more important when discussing issues as morally fraught as torture, that we don't open our minds so far that our souls fall out.

Well said.

Posted by: Eliezer Yudkowsky | September 16, 2007 at 06:07 PM"

What does that even mean? The quote and concept seem to distract from empirical inquiry and rational problem solving, rather than add to them. I'm not sure I see it as different in kind from "If it doesn't fit, then you must acquit."

HA, this is not the forum for it but perhaps somewhere else you might explain where you stand.

The way US law used to work, you or anybody else got to torture whenever you thought it was a good idea, and then you could expect to stand trial. If the police thought you did the right thing they might choose not to investigate, if the DA thought you did the right thing he might not prosecute, if the grand jury thought you did the right thing they might not continue, if the jury thought you did the right thing they might not convict, if the judge thought you did... (read more)

Tom Myers,
I think the convention on this blog, among the small set of people who have such a precise definition, is that not every heuristic is a bias, that only heuristics whose errors are worth overcoming should be called biases. I don't like this usage. For one thing, it's really hard to know the cost of overcoming particular biases. It's easy to look at an isolated experiment and say: here's a procedure that would have been better, but that doesn't cover cost of actually changing your behavior to look out for this kind of error.

Also, there are other ... (read more)

mtraven, do you really believe in the existence of the soul, or are you just using it because it is in common usage? At my blog I was thinking of writing a post whose title began "Thank god", then remembered I had already declared I was an agnotheist, and then considered "Thank goodness", but remembered I didn't believe in objective good either.

Do not take my rhetorical flourishes overly literally. We may be meat machines without an inhabiting ghost, but that doesn't mean that "soul" doesn't have meaningful connotations. In this context, it (roughly) means "our deep-seated representations of ourselves, and others, and morality, and our ability to empathize with others." It means that a rationality that only considers your own personal ends (as HA seems to advocate) is deficient. It means that people who blithely talk about torture as an option probably haven't spent too much... (read more)

Tom and Douglas, I'd take "bias" to mean something like "common feature of human thinking that gives us predictably wrong answers". Some biases might (for any given person) not be worth correcting, if correcting them requires sustained effort and similar effort could be more effective if deployed to other ends. I don't see why whether something's a "bias" should depend on whether it's the result of a heuristic operating out of the regime it was designed for, or on whether we can correct it.

I have the feeling that the term &quo... (read more)

Mtravern,
In my opinion you're preying on bias to achieve status advantage for yourself/posting name. But I don't think it will be as effective a social strategy in this overcomingbias medium as it would be in the general population.

Hopefully Anon, I can't imagine what great status or survival advantages you think I'm going to get by posting here. If I was interested in advancing my status I'd pick better activities than flaming pseudonymously on blogs, you may be sure.

But in a way you are right, but trivially. Presumably everybody who posts here or otherwise engages in human communication has, at some level, an underlying motive of increasing their status. Everybody hopes to make themselves look good. I don't see how my postings differ from anybody else's in this regard.

mtravern,
I'll have a response up to your post on my blog within 10 minutes.

http://www.hopeanon.typepad.com

getting out of touch with your basic humanity
I am a homo sapien, therefore my characteristics are human. Perhaps I should wonder why you have an inhuman bias against torture, but of course that is human as well.
Homo sum: humani nil a me alienum puto

Douglas, with regard to systematic but unexplained errors, would you agree that we can (usually) describe these as due to unidentified heuristics? I'm feeling very unsure about that, but I would like to have some fairly concrete way of thinking about this blog's subject matter, and at least this way it's something I've taught. :-) I'm not about to insist that all thought can be modeled with symbol-processing. It may even be that the most fundamental errors are those that arise without any symbol-processing -- I've just been reading a dog-trainer's book whi... (read more)

Tom Myers,
Systematic but unexplained: sure, most errors are probably due to heuristics, but I'm not sure that's a useful statement. A number of posts here have been so specific, they don't seem like useful starting points for searching for heuristics.

Cost:
Most people don't seem to have sufficiently clear goals to make sense of whether something benefits or costs them, let alone balancing the two.
People live normal lives by not thinking too much about things, so it shouldn't be so surprising that they don't think in psych experiments in which it is ofte... (read more)

Douglas Knight: for me, thinking of "bias" (as used on this blog) as a result of heuristic processing is moderately useful 'cos (a) mainly, it just gives a general framework, a set of very concrete metaphors and therefore heuristics (and therefore biases) that I've worked with over the years; (b) it suggests that the problem of bias can be ameliorated but not solved, because you'll never get perfect heuristics and you'll never be able to do all the computing that's required to do without heuristics; (c) ah, well, I forget what (c) was gonna be. B... (read more)

You probably won't go far wrong if you assume I agree with you on the points I don't respond to. I probably shouldn't have talked about them in the first place.

overcoming heuristics:
If we know a bias is caused by a heuristic, then we should use that heuristic less. But articulating a meta-heuristic about when to use it is very different implementing that meta-heuristic. Human minds aren't eurisko that can dial up the strength on heuristics. Even if we implement a heuristic, as in Kahneman's planning anecdote, and get more accurate information, we may simp... (read more)

I guess we're not disagreeing about much, at this point, though I think that you're basically more optimistic than I, and this might cause us to form different conceptions of the "overcoming bias" enterprise. I agree that we're not Eurisko (and suddenly I'm remembering Lenat's talk at IJCAI-77, explaining AM's fixed-heuristics problem that then led him to Eurisko...I was a graduate student) but my feeling is that we don't in general even have the choice of using a given heuristic less: we don't in general have the choice of becoming a less initia... (read more)

:The German text of the taped police examination, each page corrected and approved by EIchmann, constitutes a veritable gold mine for a psychologist - provided he is wise enough to understand that the horrible can be not only ludicrous but outright funny. Some of the comedy cannot be conveyed in English, because it lies in Eichmann's heroic fight with the Germna language, which invariably defeats him. It is funny when he speaks, passim, of "winged words" (geflugelte Worte, a Gemran colloquialism of famous quotes from the classics) when we means s... (read more)

Clearly, Winston was just an actor, and O'Brien was being tested by a researcher at Miniluv, whose name was Stanley Milgram.

I find it eerie how intellectually close O'Brien's mindset is to that of the pure Post-Modernist of today (for the purpose of comparison, we will disregard the fact that the word 'pure' in relation to post-modernism is an oxymoron). If a meteor smashed through a roof, someone then cleaned up the mess and patched a hole in the roof, then erased the memories of those in proximity at the time, then the meteor never fell. This is exactly how O'Brien deals with Winston's refusal to depart from the bare physicality of arithmetic on the fingers; the key similar... (read more)

New Speak always chilled me. To limit the abilities of population by limiting their vocabulary seems like a sneaky and underhanded thing to do that should never be attempted by any government. Needless to say the idea of 1984 being real freaks me out.