I think a better way to frame this issue would be the following method.

1. Present your philosophical thought-experiment.
2. Ask your subject for their response and their justification.
3. Ask your subject, what would need to change for them to change their belief?

For example, if I respond to your question of the solitary traveler with "You shouldn't do it because of biological concerns." Accept the answer and then ask, what would need to change in this situation for you to accept the killing of the traveler as moral?

I remember this method giving me deeper insight into the Happiness Box experiment.

Here is how the process works:

1. There is a happiness box. Once you enter it, you will be completely happy through living in a virtual world. You will never leave the box. Would you enter it?
2. Initial response. Yes, I would enter the box. Since my world is only made up of my perceptions of reality, there is no difference between the happiness box and the real world. Since I will be happier in the happiness box, I would enter.
3. Reframing question. What would need to change so you would not enter the box.
4. My response: Well, if I had children or people depending on me, I could not enter.

Surprising conclusion! Aha! Then you do believe that there is a difference between a happiness box and the real world, namely your acceptance of the existence of other minds and the obligations those minds place on you.

That distinction was important to me, not only intellectually but in how I approached my life.

Hope this contributes to the conversation.

David

You need to choose a very carefully targeted charity that takes into account knock-on effects like displacing local production, before you should accept any such claim. It is now being suggested that the majority of aid is actually counterproductive and destroys local industry in exchange for giving Westerners a warm fuzzy feeling. Hence Africa staying poor despite (because of?) hundreds of billions spent.

Frankly, if you're going to put in that much work into buying expected utilons at the cheapest available price, you shouldn't be considering any mainstream charity targets like aid to developing countries. Let Bill Gates worry about it, if that's the limit of his creativity.

That both women and men are far happier living with traditional gender roles. That modern Western women often hold very wrong beliefs about what will make them happy, and have been taught to cling to these false beliefs even in the face of overwhelming personal evidence that they are false.

I’ll describe three most interesting cases.

Number One is a Russian guy, now in his late 40s, with a spectacular youth. Among his trades were smuggling (during the Soviet era he smuggled brandy from Kazakhstan to Russia in the water system of a railway car), teaching in a ghetto college (where he inadvertently tamed a class of delinquents by hurling a wrench at their leader), leading a programming lab in an industrial institute, starting the first 3D visualization company in our city, reselling TV advertising time at a great margin (which he obtained by undercover deals involving key TV people and some outright gangsters), and saving the world by trying to find venture funding for a savant inventor who supposedly had a technology enabling geothermal energy extraction (I also worked together with them on this project). He was capable of totally crazy things, such as harpooning a wall portrait of a notorious Caucasus clanlord in a room full of his followers. He had lots of money during his successful periods, but was unable to convert this into a longer-term success.

Number Two is a deaf-mute woman, now in her 40s, who owns and runs a web development company. Her speech is distorted, she reads people by the lips, and I wouldn’t rate her particularly attractive – but despite all this she is able to consistently score excellent development / outsourcing contracts with top nationwide brands. Unfortunately, she often forces totally weird decisions upon the development team – and when they try to convince her that the decisions are rubbish by appealing to ‘facts’ and ‘reality’, she takes it as a direct attack on her status. A real example – she once actually imposed an official ban on criticizing her company, decisions of her company, employees and management of her company, partners of her company, products of her company and everything else directly or indirectly related to her company in all communication channels (Skype, bug trackers, IMs, phone conversations, forums etc.)!

Number Three is the most spectacular one – a Russian guy of Jewish descent, around 30, an avid status seeker with an alpha-male attitude who owns and runs several web / game outsourcing companies, plus has a high-level, high-status management/consultancy job in a well-known nationwide online company. He is almost always able to somehow secure funding for his companies and projects, including those which I personally wouldn’t consider marketable. He lives in several cities at once, is excellent at remote leadership and hiring, and is quick to act – when he learns of a talented programming or art team he could potentially partner with, he gets on a plane just to meet them in person.

It was this guy who made me seriously wonder about how immensely weird people’s worldviews can be. This guy constructs his worldview by cherry-picking pieces that appeal to his sense of truth from Abrahamic religions (excluding Islam and the Old Testament), Eastern teachings and, if I remember correctly, even fiction. He hates concepts like ‘logic’, ‘science’, ‘fact’ and ‘reality’ with a passion, and believes that they are evil concepts designed by Anglo-Saxons to corrode ‘good’ worldviews (such as the New-Testament Christian one), and he is actively protecting his worldview from being corroded by evil ideas. Here’s an actual example of his reasoning: “Dawkins is an Anglo-Saxon, and all Anglo-Saxons are evil liars, therefore all ideas Dawkins advocates are evil lies, therefore evolution is a lie and is evil.” He believes that The Enemy himself is sponsoring the evolutionary science by actually providing money, fame and other goods to its proponents. He is sincerely unable to understand how people can be genuinely altruistic without a religious upbringing (and of course, he doesn’t want to consider things like mirror neurons).

So, it was this guy who made me ask myself questions like ‘What is my definition of truth?’

One of the reasons parents often give for the Santa myth is that it is "fun" and it's good to give children a sense of wonder and joy. This is not a trivial argument.

I don't have children yet, but this post has made me wonder if a strictly no-lies-about-basic-reality policy wouldn't lead to just as much wonder. Is Santa really that necessary, even for the stated junior-level purpose it's given?

There are lots of fantastic, amazing things to wonder at, as a child and as an adult:

Everything in our universe may have started in the biggest, most gigantic-est explosion ever! BLAM!

There are millions of tiny critters living inside your tummy right now and they are helping you every day to eat your food! YUM!

All 7 billion people in the world are the great-great-great-(....)great-great grandchildren of one large-sized family of people who came from Africa! WOW!

I cannot forgive the incorrect subargument, "It's worse than being religious."

I'm not sure how you could misconstrue that as an argument - it's a single proposition!

If it's not obvious, consider that they're rejecting a commonly-held belief for a really bad reason. That's practically insane. Much better to go along with the crowd until you have an actual reason not to.

And I think being neither rude nor a jerk is vital to being a complete person. You're setting the bar way too low.

Pjeby's working on akrasia? I'll have to check out his site.

That brings up a related question that I think Eliezer hinted at: what pre-existing bodies of knowledge can we search through for powerful techniques so that we don't have to re-invent the wheel? Entrepreneurship stuff is one. Lots of people have brought up pick-up artists and poker, so those might be others.

I nominate a fourth that may be controversial: mysticism. Not the "summon demons" style of mysticism, but yoga and Zen and related practices. These people have been learning how to examine/quiet/rearrange their minds and sort out the useful processes from the useless processes for the past three thousand years. Even if they've been working off crazy metaphysics, it'd be surprising if they didn't come up with something. Eliezer talks in mystical language sometimes, but I don't know whether that's because he's studied and approves of mysticism or just likes the feel of it.

What all of these things need is a testing process combined with people who are already high-level enough that they can sort through all the dross and determine which techniques are useful without going native or opening themselves up to the accusation that they're doing so; ie people who can sort through the mystical/pick-up artist/whatever literature and separate out the things that are useful to rationalists from the things specific to a certain worldview hostile to our own. I've seen a few good people try this, but it's a mental minefield and they tend to end up "going native".

Why aren't "rationalists" surrounded by a visible aura of formidability? Why aren't they found at the top level of every elite selected on any basis that has anything to do with thought? Why do most "rationalists" just seem like ordinary people, perhaps of moderately above-average intelligence, with one more hobbyhorse to ride?

Because they don't win? Because they don't reliably steer reality into narrow regions other people consider desirable?

I've met and worked with several irrationalists whose models of reality were, to put it mildly, not correlated to said reailty, with one explicit, outspoken anti-rationalist with a totally weird, alien epistemology among them. All these people had a couple of interesting things in common.

On one hand, they were often dismal at planning – they were unable to see obvious things, and they couldn't be convinced otherwise by any arguments appealing to 'facts' and 'reality' (they universally hated these words).

On the other hand, they were surprisingly good at execution. All of them were very energetic people who didn't fear any work or situation at all, and I almost never saw any of them procrastinating. Could this be because their minds, due to their poor predictive ability, were unable to see the real difficulty of their tasks and thus avoided auto-switching into procrastination mode?

(And a third observation – all these people excelled in political environments. They tended to interpret their surroundings primarily in terms of who is kin to whom, who is a friend of who, who is sexually attracted to whom, what others think of me, who is the most influential dude around here etc etc. What they lost due to their desynchronization with factual reality, they gained back thanks to their political aptness. Do rationalists excel in political environments?)

Suppose I present a concrete non-rigorous analogy: "A chain letter is like an organism with a habitat of human minds.". What is the easy challenge that I have left myself open to? I already freely conceded that it was non-rigorous.

Neither do I, though I'm often tempted to find a reason for why my iPod's shuffle function "chose" a particular song at a particular time. ["Mad World" right now.]

It seems that our mental 'hardware' is very susceptible to agency and causal misfires, leaving an opening for something like religious belief. Robin explained religious activities and beliefs as important in group bonding [http://www.overcomingbias.com/2009/01/why-fiction-lies.html], but the fact that religion arose may just be a historical accident. It's likely that something would have arisen in the same place as a group bonding mechanism - perhaps religion just found the gap first. From an individual perspective, this hardly means that the sanity waterline is low. In fact, evolutionarily speaking, playing along may be the sanest thing to do.

The relevant sentence from Robin's post: "Social life is all about signaling our abilities and cooperativeness, and discerning such signals from others." As Norman points out [link below], self-deception makes our signals more credible, since we don't have to act as believers if we are believers. As a result, in the ancestral environment at least, it's "sane" to believe what others believe and not subject it to a conscious and costly rationality analysis. You'd basically expend resources to find out a truth that would make it more difficult for me to deceive others, which is costly in itself.

Of course today, the payoff from signaling group membership is far lower than ever before, which is why religious belief, and especially costly religious activities, violate sanity. Which, perhaps, is why secularism is on the rise: http://www.theatlantic.com/doc/200803/secularism