I suspect the real problem with the lottery is that people are familiar with the amounts of money necessary to purchase a ticket and recognize that it won't bring them much happiness or satisfaction... but they're not familiar with the amounts of money typically given away in a lottery jackpot, and they imagine that it will make them much, much happier than it is actually likely too.

Even people who know that the lottery has a negative expected financial value buy tickets. They'll accept a resource loss if it leads to greater utility, and people tend to perceive the slim chance of winning lots of money to be sufficiently valuable that they'll take an expected loss to have the chance.

The irrationality comes not from dismissing the statistical loss of money, but from believing that winning offers much greater utility than it will.

Assuming spontaneous original thought is too difficult (and I doubt anything in this comment is original), how about this as a ritualised way of avoiding group-think:

A company has regular meetings to discuss its tactics. However, before the meeting, the boss tells one of the participants to be a rebel. (The others don't know who is the designated rebel at a given meeting, but it is understood that everyone will be told to play rebel sooner or later, for fairness if nothing else.) The rebel's job is to come up with persuasive arguments against the consensus position, even if it's a consensus the boss is believed to support (assuming the matter is still up for discussion). The rebel doesn't have to always take a minority position, so as not to force him into absurdities, but he has a bias in favour of rebellious behaviour because it will please the boss.

Why the secrecy? Because the uncertainty about who is the rebel creates a window for other participants to genuinely express anti-consensus opinions, something they'd otherwise be afraid to do for fear of ostracism. This is the real purpose of the rebel from the perspective of the boss.

Now the danger here is that the designated rebel will come to the meeting wearing black, so to speak, and so won't actually count for much in the social perspective of the other participants. However, the rebel has an incentive not to make it so obvious. In fact, even the would-be conformists benefit from disguising the rebel, if they think the consensus is genuinely the right position, because as soon as the rebel is unmasked, the aura of the boss is also clear to see on him, so others would be socially obliged to show him more respect. (But this could make them inclined to agree with him, which makes his job as rebel intellectually taxing, coupled with the extra pressure of increased attention, so he won't enjoy this reverence too much.) Also, the others may feel sympathy for the rebel, because it's not a role he has chosen, and the chances are that they will be called on to do the same. This sympathy also extends to *possible* rebels. So this will hopefully make dissent much more socially acceptable, and reduce the urge to 'destroy the traitor' by ignoring or ridiculing him.

Why have only one rebel? Because if everyone were rewarded for rebellion, it would create constant disagreement for the sake of it. (To make it clear, the boss does *not* automatically reward all rebellions in the meeting, only those of the designated rebel.) The designated rebel is just there to break the spell of unanimity. He is made to sacrifice much of his own freedom of action, but in a decent-sized group this is hopefully compensated for by the increased independence of the others.

Does this kind of manufactured dissent actually work in reducing bias overall? Or would the 'we hate the lone rebel' bias prove too strong to overcome, even when it's theoretically trumped by the approval of the boss?

A surprising number of people I meet seem to know exactly how to build an Artificial General Intelligence, without, say, knowing how to play the guitar or juggle (much easier problems).

I totally agree, Eliezer. Yet I like making references to science fiction when I discuss the future when discussing with friends, or on my blog for a couple of reasons:

- It's a strong argument in favor of accelerating change: the technology that exists today is way beyond many of the gadgets depicted in SF from a few decades back which predicted them for 1500 years later. And, even more impressing is that these gadgets are cheap and available to anyone, at least in rich countries (mobile phones, the Web, GPS, iPods...). If anything, it stresses how common wisdom downplays the evolution of technologies, which helps to make a case for AGI emerging in decades, not centuries.

- SF helps to raise important questions about the future which are hard to address in the setting of the present. The classic example of that is the failure of Asimov's law of robotics. The more recent example is the TV series BattleStar Galactica. Of course it's unrealistic and biased, but it changed my views on the issues of AGI's rights. Can a robot be destroyed without a proper trial? Is it OK to torture it? to rape it? What about marrying one? or having children with it (or should I type "her")?

I had to look at the html source where you said "Try to say aloud the color - not the meaning, but the color - of the following letter-string: "GREEN"" because I'm colorblind and I couldn't tell what color it was. Small amounts of red or green appear to be BOTH red and green simultaneously haha (show me a giant field of green and I can tell it's green most of the time, but show me a dot of green on a field of white and I have no clue, same with red). I guess that really isn't relevant to anything said here, I just thought it was funny considering the point of the exercise.

"They usually resort to the script of presuming a personal insult" instead of rightly apprehending the point you're making, which is...?

This is the difficulty I have with your comments, Caledonian. You always leave the interesting part out. (This is not a personal insult, by the way -- just a straightforward observation.)

Really, Chris. So if I believe in the value of the environment, but believe that it's much less valuable than the use to be gained by paving it over with strip mines, then I'm an "environmentalist"?

In any case it's a moot point. Mark Plus coined the term "Singularitarian", but didn't do much with it; when I decided to build a Singularitarian movement, I asked Mark Plus for ownership of the word and was granted it; and I define the term to involve activism. If you mean something else by the word, feel free to call yourself a "Singularian" or something.

Can someone just tell us dumb asses the differece between describing something and experiencing it?

Um... ok.

Description: If you roll your face on your keyboard you will feel the keys mushing and pressing against your face. The most pronounced features of the tactile experience will be the feeling of the ridges of the keys pressing against your forehead, eyebrows and cheekbones. You will also hear a subtle "thrumping" noise of the keys are being pressed. If you didn't put the cursor in a text editor you might hear some beeps from your computer. Once you lift your head you may still have some residual sensations on your face most likely where the relatively sharp ridges of the keys came in contact with your skin.

Experience: Roll your face on your keyboard. Don't just read this, you have to actually roll your face on the keyboard if you want to experience it. 1, 2, 3, go ... bnkiv7n6ym7n9t675r

Did you notice any difference between the description and the experience?

Anyways, I still hold that you can only define reductionism up to point after which you are just wasting time.