The Scales of Justice, the Notebook of Rationality

27Eliezer_Yudkowsky13 March 2007 04:00PM

Lady Justice is widely depicted as carrying a scales.  A scales has the property that whatever pulls one side down, pushes the other side up.  This makes things very convenient and easy to track.  It's also usually a gross distortion.

In human discourse there is a natural tendency to treat discussion as a form of combat, an extension of war, a sport; and in sports you only need to keep track of how many points have been scored by each team.  There are only two sides, and every point scored against one side, is a point in favor of the other.  Everyone in the audience keeps a mental running count of how many points each speaker scores against the other.  At the end of the debate, the speaker who has scored more points is, obviously, the winner; so everything he says must be true, and everything the loser says must be wrong.

continue reading »

Knowing About Biases Can Hurt People

41Eliezer_Yudkowsky04 April 2007 06:01PM

Once upon a time I tried to tell my mother about the problem of expert calibration, saying:  "So when an expert says they're 99% confident, it only happens about 70% of the time."  Then there was a pause as, suddenly, I realized I was talking to my mother, and I hastily added:  "Of course, you've got to make sure to apply that skepticism evenhandedly, including to yourself, rather than just using it to argue against anything you disagree with—"

And my mother said:  "Are you kidding?  This is great!  I'm going to use it all the time!"

continue reading »

Lotteries: A Waste of Hope

20Eliezer_Yudkowsky13 April 2007 05:36AM

The classic criticism of the lottery is that the people who play are the ones who can least afford to lose; that the lottery is a sink of money, draining wealth from those who most need it.  Some lottery advocates, and even some commentors on this blog, have tried to defend lottery-ticket buying as a rational purchase of fantasy—paying a dollar for a day's worth of pleasant anticipation, imagining yourself as a millionaire.

But consider exactly what this implies.  It would mean that you're occupying your valuable brain with a fantasy whose real probability is nearly zero—a tiny line of likelihood which you, yourself, can do nothing to realize.  The lottery balls will decide your future.  The fantasy is of wealth that arrives without effort—without conscientiousness, learning, charisma, or even patience.

Which makes the lottery another kind of sink: a sink of emotional energy.  It encourages people to invest their dreams, their hopes for a better future, into an infinitesimal probability.  If not for the lottery, maybe they would fantasize about going to technical school, or opening their own business, or getting a promotion at work—things they might be able to actually do, hopes that would make them want to become stronger.  Their dreaming brains might, in the 20th visualization of the pleasant fantasy, notice a way to really do it.  Isn't that what dreams and brains are for?  But how can such reality-limited fare compete with the artificially sweetened prospect of instant wealth—not after herding a dot-com startup through to IPO, but on Tuesday?

continue reading »

Scope Insensitivity

29Eliezer_Yudkowsky14 May 2007 02:53AM

Once upon a time, three groups of subjects were asked how much they would pay to save 2000 / 20000 / 200000 migrating birds from drowning in uncovered oil ponds. The groups respectively answered $80, $78, and $88 [1]. This is scope insensitivity or scope neglect: the number of birds saved - the scope of the altruistic action - had little effect on willingness to pay.

Similar experiments showed that Toronto residents would pay little more to clean up all polluted lakes in Ontario than polluted lakes in a particular region of Ontario [2], or that residents of four western US states would pay only 28% more to protect all 57 wilderness areas in those states than to protect a single area [3].

continue reading »

One Life Against the World

23Eliezer_Yudkowsky18 May 2007 10:06PM

Followup to: Scope Insensitivity

"Whoever saves a single life, it is as if he had saved the whole world."

-- The Talmud, Sanhedrin 4:5

It's a beautiful thought, isn't it? Feel that warm glow.

I can testify that helping one person feels just as good as helping the whole world. Once upon a time, when I was burned out for the day and wasting time on the Internet - it's a bit complicated, but essentially, I managed to turn someone's whole life around by leaving an anonymous blog comment. I wasn't expecting it to have an effect that large, but it did. When I discovered what I had accomplished, it gave me a tremendous high. The euphoria lasted through that day and into the night, only wearing off somewhat the next morning. It felt just as good (this is the scary part) as the euphoria of a major scientific insight, which had previously been my best referent for what it might feel like to do drugs.

Saving one life probably does feel just as good as being the first person to realize what makes the stars shine. It probably does feel just as good as saving the entire world.

But if you ever have a choice, dear reader, between saving a single life and saving the whole world - then save the world. Please. Because beyond that warm glow is one heck of a gigantic difference.

continue reading »

Risk-Free Bonds Aren't

10Eliezer_Yudkowsky22 June 2007 10:30PM

I've always been annoyed by the term "risk-free bonds rate", meaning the return on US Treasury bills.  Just because US bonds have not defaulted within their trading experience, people assume this is impossible?  A list of major governments in 1900 would probably put the Ottoman Empire or Austria-Hungary well ahead of the relatively young United States.  Citing the good track record of the US alone, and not all governments of equal apparent stability at the start of the same time period, is purest survivorship bias.

The United States is a democracy; if enough people vote for representatives who decide not to pay off the bonds, they won't get paid.  Do you want to look at recent history, let alone ancient history, and tell me this is impossible?  The Internet could enable coordinated populist voting that would sweep new candidates into office, in defiance of prevous political machines.  Then the US economy melts under the burden of consumer debt, which causes China to stop buying US bonds and dump its dollar reserves.  Then Al Qaeda finally smuggles a nuke into Washington, D.C.  Then the next global pandemic hits.  And these are just "good stories" - the probability of the US defaulting on its bonds for any reason, is necessarily higher than the probability of it happening for the particular reasons I've just described.  I'm not saying these are high probabilities, but they are probabilities.  Treasury bills are nowhere near "risk free". 

continue reading »

Correspondence Bias

27Eliezer_Yudkowsky25 June 2007 12:58AM

The correspondence bias is the tendency to draw inferences about a person's unique and enduring dispositions from behaviors that can be entirely explained by the situations in which they occur.
     —Gilbert and Malone

We tend to see far too direct a correspondence between others' actions and personalities.  When we see someone else kick a vending machine for no visible reason, we assume they are "an angry person".  But when you yourself kick the vending machine, it's because the bus was late, the train was early, your report is overdue, and now the damned vending machine has eaten your lunch money for the second day in a row.  Surely, you think to yourself, anyone would kick the vending machine, in that situation.

continue reading »

Are Your Enemies Innately Evil?

57Eliezer_Yudkowsky26 June 2007 09:13PM

Followup to:  Correspondence Bias

As previously discussed, we see far too direct a correspondence between others' actions and their inherent dispositions.  We see unusual dispositions that exactly match the unusual behavior, rather than asking after real situations or imagined situations that could explain the behavior.  We hypothesize mutants.

When someone actually offends us—commits an action of which we (rightly or wrongly) disapprove—then, I observe, the correspondence bias redoubles.  There seems to be a very strong tendency to blame evil deeds on the Enemy's mutant, evil disposition.  Not as a moral point, but as a strict question of prior probability, we should ask what the Enemy might believe about their situation which would reduce the seeming bizarrity of their behavior.  This would allow us to hypothesize a less exceptional disposition, and thereby shoulder a lesser burden of improbability.

On September 11th, 2001, nineteen Muslim males hijacked four jet airliners in a deliberately suicidal effort to hurt the United States of America.  Now why do you suppose they might have done that?  Because they saw the USA as a beacon of freedom to the world, but were born with a mutant disposition that made them hate freedom?

continue reading »

Hindsight bias

38Eliezer_Yudkowsky16 August 2007 09:58PM

Hindsight bias is when people who know the answer vastly overestimate its predictability or obviousness, compared to the estimates of subjects who must guess without advance knowledge.  Hindsight bias is sometimes called the I-knew-it-all-along effect.

Fischhoff and Beyth (1975) presented students with historical accounts of unfamiliar incidents, such as a conflict between the Gurkhas and the British in 1814.  Given the account as background knowledge, five groups of students were asked what they would have predicted as the probability for each of four outcomes: British victory, Gurkha victory, stalemate with a peace settlement, or stalemate with no peace settlement.  Four experimental groups were respectively told that these four outcomes were the historical outcome.  The fifth, control group was not told any historical outcome.  In every case, a group told an outcome assigned substantially higher probability to that outcome, than did any other group or the control group.

continue reading »

Hindsight Devalues Science

56Eliezer_Yudkowsky17 August 2007 07:39PM

This excerpt from Meyers's Exploring Social Psychology is worth reading in entirety.  Cullen Murphy, editor of The Atlantic, said that the social sciences turn up "no ideas or conclusions that can't be found in [any] encyclopedia of quotations... Day after day social scientists go out into the world.  Day after day they discover that people's behavior is pretty much what you'd expect."

Of course, the "expectation" is all hindsight.  (Hindsight bias:  Subjects who know the actual answer to a question assign much higher probabilities they "would have" guessed for that answer, compared to subjects who must guess without knowing the answer.)

The historian Arthur Schlesinger, Jr. dismissed scientific studies of WWII soldiers' experiences as "ponderous demonstrations" of common sense.  For example:

  1. Better educated soldiers suffered more adjustment problems than less educated soldiers. (Intellectuals were less prepared for battle stresses than street-smart people.)  
  2. Southern soldiers coped better with the hot South Sea Island climate than Northern soldiers. (Southerners are more accustomed to hot weather.)  
  3. White privates were more eager to be promoted to noncommissioned officers than Black privates. (Years of oppression take a toll on achievement motivation.)  
  4. Southern Blacks preferred Southern to Northern White officers (because Southern officers were more experienced and skilled in interacting with Blacks).  
  5. As long as the fighting continued, soldiers were more eager to return home than after the war ended. (During the fighting, soldiers knew they were in mortal danger.)  

How many of these findings do you think you could have predicted in advance?   3 out of 5?  4 out of 5?  Are there any cases where you would have predicted the opposite—where your model takes a hit?  Take a moment to think before continuing...

continue reading »

View more: Next