Many Christians who've stopped really believing now insist that they revere the Bible as a source of ethical advice.  The standard atheist reply is given by Sam Harris:  "You and I both know that it would take us five minutes to produce a book that offers a more coherent and compassionate morality than the Bible does."  Similarly, one may try to insist that the Bible is valuable as a literary work.  Then why not revere Lord of the Rings, a vastly superior literary work?  And despite the standard criticisms of Tolkien's morality, Lord of the Rings is at least superior to the Bible as a source of ethics.  So why don't people wear little rings around their neck, instead of crosses?  Even Harry Potter is superior to the Bible, both as a work of literary art and as moral philosophy.  If I really wanted to be cruel, I would compare the Bible to Jacqueline Carey's Kushiel series.

"How can you justify buying a $1 million gem-studded laptop," you ask your friend, "when so many people have no laptops at all?"  And your friend says, "But think of the employment that this will provide—to the laptop maker, the laptop maker's advertising agency—and then they'll buy meals and haircuts—it will stimulate the economy and eventually many people will get their own laptops."  But it would be even more efficient to buy 5,000 OLPC laptops, thus providing employment to the OLPC manufacturers and giving out laptops directly.

I've touched before on the failure to look for third alternatives.  But this is not really motivated stopping.  Calling it "motivated stopping" would imply that there was a search carried out in the first place.

"What's the worst that can happen?" goes the optimistic saying.  It's probably a bad question to ask anyone with a creative imagination.  Let's consider the problem on an individual level: it's not really the worst that can happen, but would nonetheless be fairly bad, if you were horribly tortured for a number of years.  This is one of the worse things that can realistically happen to one person in today's world.

What's the least bad, bad thing that can happen?  Well, suppose a dust speck floated into your eye and irritated it just a little, for a fraction of a second, barely enough to make you notice before you blink and wipe away the dust speck.

For our next ingredient, we need a large number.  Let's use 3^^^3, written in Knuth's up-arrow notation:

• 3^3 = 27.
• 3^^3 = (3^(3^3)) = 3^27 = 7625597484987.
• 3^^^3 = (3^^(3^^3)) = 3^^7625597484987 = (3^(3^(3^(... 7625597484987 times ...)))).

3^^^3 is an exponential tower of 3s which is 7,625,597,484,987 layers tall.  You start with 1; raise 3 to the power of 1 to get 3; raise 3 to the power of 3 to get 27; raise 3 to the power of 27 to get 7625597484987; raise 3 to the power of 7625597484987 to get a number much larger than the number of atoms in the universe, but which could still be written down in base 10, on 100 square kilometers of paper; then raise 3 to that power; and continue until you've exponentiated 7625597484987 times.  That's 3^^^3.  It's the smallest simple inconceivably huge number I know.

Now here's the moral dilemma.  If neither event is going to happen to you personally, but you still had to choose one or the other:

Would you prefer that one person be horribly tortured for fifty years without hope or rest, or that 3^^^3 people get dust specks in their eyes?

I think the answer is obvious.  How about you?

(This is the third post in a planned sequence.)

My last post left us with the questions:

• Just what are humans, and other common CSAs, calculating when we imagine what “would” happen “if” we took actions we won’t take? 
• Is there more than one natural way to calculate these counterfactual “would”s?  If so, what are the alternatives, and which alternative works best?

Today, I’ll take an initial swing at these questions.  I’ll review Judea Pearl’s causal Bayes nets; show how Bayes nets offer a general methodology for computing counterfactual “would”s; and note three plausible alternatives for how to use Pearl’s Bayes nets to set up a CSA.  One of these alternatives will be the “timeless” counterfactuals of Eliezer’s Timeless Decision Theory.

Followup to:  The Third Alternative, The Meditation on Curiosity

While I disagree with some views of the Fast and Frugal crowd—IMO they make a few too many lemons into lemonade—it also seems to me that they tend to develop the most psychologically realistic models of any school of decision theory.  Most experiments present the subjects with options, and the subject chooses an option, and that's the experimental result.  The frugalists realized that in real life, you have to generate your options, and they studied how subjects did that.

Likewise, although many experiments present evidence on a silver platter, in real life you have to gather evidence, which may be costly, and at some point decide that you have enough evidence to stop and choose.  When you're buying a house, you don't get exactly 10 houses to choose from, and you aren't led on a guided tour of all of them before you're allowed to decide anything.  You look at one house, and another, and compare them to each other; you adjust your aspirations—reconsider how much you really need to be close to your workplace and how much you're really willing to pay; you decide which house to look at next; and at some point you decide that you've seen enough houses, and choose.

Followup to:  Explainers Shoot High, Illusion of Transparency

My first true foray into Bayes For Everyone was writing An Intuitive Explanation of Bayesian Reasoning, still one of my most popular works.  This is the Intuitive Explanation's origin story.

In December of 2002, I'd been sermonizing in a habitual IRC channels about what seemed to me like a very straightforward idea:  How words, like all other useful forms of thought, are secretly a disguised form of Bayesian inference.  I thought I was explaining clearly, and yet there was one fellow, it seemed, who didn't get it.  This worried me, because this was someone who'd been very enthusiastic about my Bayesian sermons up to that point.  He'd gone around telling people that Bayes was "the secret of the universe", a phrase I'd been known to use.

So I went into a private IRC conversation to clear up the sticking point.

Sometime between the age of 3 and 4, a human child becomes able, for the first time, to model other minds as having different beliefs.  The child sees a box, sees candy in the box, and sees that Sally sees the box.  Sally leaves, and then the experimenter, in front of the child, replaces the candy with pencils and closes the box so that the inside is not visible.  Sally returns, and the child is asked what Sally thinks is in the box.  Children younger than 3 say "pencils", children older than 4 say "candy".

Our ability to visualize other minds is imperfect.  Neural circuitry is not as flexible as a program fed to a general-purpose computer.  An AI, with fast read-write access to its own memory, might be able to create a distinct, simulated visual cortex to imagine what a human "sees".  We humans only have one visual cortex, and if we want to imagine what someone else is seeing, we've got to simulate it using our own visual cortex - put our own brains into the other mind's shoes.  And because you can't reconfigure memory to simulate a new brain from stratch, pieces of you leak into your visualization of the Other.

In hindsight bias, people who know the outcome of a situation believe the outcome should have been easy to predict in advance.  Knowing the outcome, we reinterpret the situation in light of that outcome.  Even when warned, we can't de-interpret to empathize with someone who doesn't know what we know.

Closely related is the illusion of transparency:  We always know what we mean by our words, and so we expect others to know it too.  Reading our own writing, the intended interpretation falls easily into place, guided by our knowledge of what we really meant.  It's hard to empathize with someone who must interpret blindly, guided only by the words.

June recommends a restaurant to Mark; Mark dines there and discovers (a) unimpressive food and mediocre service (b) delicious food and impeccable service.  Then Mark leaves the following message on June's answering machine:  "June, I just finished dinner at the restaurant you recommended, and I must say, it was marvelous, just marvelous."  Keysar (1994) presented a group of subjects with scenario (a), and 59% thought that Mark's message was sarcastic and that Jane would perceive the sarcasm.  Among other subjects, told scenario (b), only 3% thought that Jane would perceive Mark's message as sarcastic.  Keysar and Barr (2002) seem to indicate that an actual voice message was played back to the subjects.

From pp. 55-56 of Robyn Dawes's Rational Choice in an Uncertain World.  Bolding added.

Norman R. F. Maier noted that when a group faces a problem, the natural tendency of its members is to propose possible solutions as they begin to discuss the problem.  Consequently, the group interaction focuses on the merits and problems of the proposed solutions, people become emotionally attached to the ones they have suggested, and superior solutions are not suggested.  Maier enacted an edict to enhance group problem solving: "Do not propose solutions until the problem has been discussed as thoroughly as possible without suggesting any."  It is easy to show that this edict works in contexts where there are objectively defined good solutions to problems.

Followup to:  Cached Thoughts, The Virtue of Narrowness

Since Robert Pirsig put this very well, I'll just copy down what he said.  I don't know if this story is based on reality or not, but either way, it's true.

        He'd been having trouble with students who had nothing to say. At first he thought it was laziness but later it became apparent that it wasn't.  They just couldn't think of anything to say.
        One of them, a girl with strong-lensed glasses, wanted to write a five-hundred word essay about the United States.  He was used to the sinking feeling that comes from statements like this, and suggested without disparagement that she narrow it down to just Bozeman.
       When the paper came due she didn't have it and was quite upset.  She had tried and tried but she just couldn't think of anything to say.
        It just stumped him.  Now he couldn't think of anything to say.  A silence occurred, and then a peculiar answer:  "Narrow it down to the main street of Bozeman."  It was a stroke of insight.
        She nodded dutifully and went out.  But just before her next class she came back in real distress, tears this time, distress that had obviously been there for a long time.  She still couldn't think of anything to say, and couldn't understand why, if she couldn't think of anything about all of Bozeman, she should be able to think of something about just one street.