no, no, no, you should be comparing the number of people who want to have great sex with a hot babe with the number of people who want to gain higher status. The answer for most everyone would be yes!! both! Because both were selected for by increased inclusive fitness.

I think that should be 'pretext for war', not 'pretense for war'.

Alas, even those who are governed by reason take the chance that someday a sunken fact will rip the bottom out of their boat.

The odds are only somewhat less for this happening to those governed by reason.

What is "FOOM"? Is it an acronym? What does it stand for?

Wordnik says "The sound of a muffled explosion." But that doesn't sound right. If AI goes FOOM, the explosion presumably won't be 'muffled'. :-)

This essay is wonderful. It is the first coherent and plausible defense of polytheism I've ever read. It is a much more intellectually satisfying version of creationism than monotheism.

And all just as an aside too, off the cuff. Kudos Mr. Yudkowsky. It really is a pleasure to read your work.

Seems to me you're asking the wrong question. I say, don't ask if there is a omnipotent God, that is making an unwarranted narrowing assumption. Why should it be either/or? Lots and lots of room in between the 'omnipotent God' theory and the 'no god at all' theory for 'medium potent god(s) theories.

And 'medium potent god' theories are not only inherently more likely than either of the extremes, they seem a lot more fruitful and interesting to think about, in terms of possible consequences.

I say, ask if there are beings of ANY kind that are more intelligent/knowledgeable/scientifically or morally advanced than human beings.

And, if so, are they aware of us, and interested in us, and what we do, either as individuals or as a species?

Now, that's an interesting question. If (they/he/it) exist, and are interested in our behavior, the important question isn't if (they/he/it) are the all powerful creators of the universe or not.

It's what do they want?

And, if they care about what we do, why haven't they communicated with us directly, and said (variously) in an unmistakably clear manner exactly what they do want us to do/not do? eg, flaming letters 10 miles high in the sky, urging us to (variously) not eat meat/stop burning hydrocarbons/stop committing adultery/etc/etc/etc fill in the blank. They obviously would have the technology.

I personally find thinking of Jehovah/Zeus/Odin and all their kin as advanced extraterrestrials is AT LEAST as LIKELY a theory as assuming they are real, or that they are myths. It isn't hard to believe that extraterrestrials noticed us a long time ago and are discretely guiding our progress (or discretely diverting us into blind alleys where we can't ever advance to the point where we could annoy them).

After all, the important question isn't if an omnipotent God created the universe, but if some god(s) are interested in what we do. If so, are they good/evil (from the standpoint of human morality)? Are they a threat or a blessing -- to 'mankind' or (ahem) 'the environment' (the earth and its existing ecosystem -- for those of you who belong to one of the newer religions).

Firstly because we have insufficient evidence to address the first problem. And secondly, because it doesn't make any difference in real outcomes to you and me how the universe was created. What matters to us, is if there are beings -- 'gods' -- more powerful than US. It doesn't matter if they are powerful enough to create the universe. All that matters is if they are powerful enough (and interested enough) to intervene in our lives, now or in the future (or the past).

This whole speech makes me mad. The same people who urge us to not have kids, because of overpopulation, are urging us to spend all of our disposable income in supporting 'poor people', because they are in misery. And why are they in misery? Because they had more kids than they could afford to take care of. And their parents did. And their parents before them.

You on the other hand, are descended from a long line of prudent people. Who though about the consequences of their actions and decided that the short term pleasure wasn't worth the long term pain. Those in misery, those we are being urged to save, they, and their parents, didn't exercise the self restraint to not have kids they couldn't afford, and so now we should rush in and save them -- from the pain associated with their shortsightedness. They are and were obviously more indifference to their own future suffering, and to the future suffering of their children than to the pleasure of having half a dozen kids, why should we be more concerned about their suffering than they were? You postpone marriage and sacrifice to get a good job, they go out and have 10 kids when they don't have enough food and money to support themselves.

If revealed preferences show anything, they show that they, personally, aren't actually very distressed by the threat of dying of hunger, or being debilitated by disease, or living in a society wracked by violence and injustice. And they aren't too distressed by the possibility of their children living and dying in exactly the same way. They obviously aren't concerned enough about it to change their own behavior. Why should we rush in and save them?

That's the problem with utilitarianism. Just like every other form of socialism it says that people who are smart, and conscientious, and hard working, should spend their time working to accomplish the goals of people who are lazy, stupid, and thriftless. Or rather working 24/7 to save them from the consequences of their actions. They go off and have half a dozen kids with no way to support them, and then we are supposed to spend out time maximizing "the sum total of human happiness" by supporting them and their kids -- by doing for them things they can't be bothered to do for themselves.

That's the problem with democracy too, especially democracy in the form with mass redistribution. Money (which is, literally, time) is taken from the smart, the hardworking, and the conscientious, and given to the foolish, the feckless and the irresponsible. This TRANSFERS decision making power as to what to do with the spare resources of society from people who would invest in cultural enhancement, things like scientific research, to those whose highest goals are pleasure and the shortsighted avoidance of pain.

This is a mistake on two levels:

First, it turns evolution on it's head. Soon the world is full of thriftless, heedless, irresponsible people.

Second, it destroys the possibility of progress. I say encourage smart, conscientious, hardworking people to use their spare time, energy and money to advance human KNOWLEDGE, and to have smart, conscientious, hardworking kids of their own, who will do the same.

If increasing the sum total of human happiness is your goal, there are far better ways of accomplishing it than giving all of your money to aid workers to distribute to the poor.

1) Even if you did save the lives of 10 people, by sending all your spare cash to provide food aid or medical assistance to the dreadfully poor. I don't think it actually decreases the amount of suffering in the world, or increases the amount of human happiness. It just gives 10 people you saved the 'opportunity' to suffer in a dozen other unpleasant ways. The quality of life in places where people actually die from malnutrition is dreadful.

2) If you just want to maximize the human happiness of 10 people, have 10 kids, and raise them lovingly. You'd add more to the sum total of human enjoyment in the world than someone who rescues 10 people from starvation. The number of additional " enjoyable human life years" you give them is probably nil, since you don't, and can't, make sure they are educated, or taught a useful profession, or protected from dreadful diseases, or from random violence by thugs and jihadists and the hundreds of other injustices and oppression they would encounter in Africa,or where-ever. Just making sure they don't die of starvation is just the start of the unending list of things you would have to fix to make their lives enjoyable. Cut out the middleman, those ineffectual, incompetent aid workers, who need tens of thousands of your money to marginally improve the life of one person (maybe). Raise 10 kids of your own. Guaranteed the happy, useful lives of THOSE ten kids will add more to the 'sum total of human happiness' than the lives of those hypothetical 10 people in Africa.

3) Besides, raising 10 kids of your own would dramatically increase your OWN happiness. Why feel guilty? Pleasure that you give yourself is as much a contribution to the sum total of human happiness as pleasure you enable your 10 kids experience, by living happy and productive lives. And believe me, your life would be MUCH more enjoyable. And not just from the fun of watching the kids grow up. If you're a guy, and if you're reading this blog, the odds are high you are a guy, then, believe me, there is no 'game' you can play that would be as sure a road to finding a beautiful, loving woman who wants YOU, as announcing to all and sundry that you want to get married and have a lot of kids -- and that you are looking for someone who wants that too. Try THAT the next time you walk into a bar.

4) And if you're reading this blog, the odds are high that you are about 2 standard deviations above the average human being in intelligence, conscientiousness, and willingness to work hard. If the 10 people whose lives you make possible are all YOUR KIDS, instead of some random strangers, you just helped improve the human gene pool, and by a lot. We need more people who are just like you, who care about scientific progress, and the future of the human race, etc. So get out there and replicate!! :-) ,Instead of sending your money off to help replicate other people, replicate yourself! You are exactly the sort of people we need more of.

Even better, if you do have kids, (instead of spending your time supporting other peoples kids), you can raise them to be USEFUL: scientists, engineers, researchers -- people who can make a real contribution to humanity, instead of eaking out their lives as subsistence agriculturalists, or as "marginally attached workers" in some slum of a mega city, or idling their time away in some refugee camp...

The problem here is that you don't KNOW that the probability is 90%. What if it's 80%? or 60%? or 12%? In real life you will only run the experiment once. The probabilities are just a GUESS. The person who is making the guess has no idea what the real probabilities are. And as Mr. Yudkowsky has pointed out elsewhere, people consistently tend to underestimate the difficulty of a task. They can't even estimate with any accuracy how long it will take them to finish their homework. If you aren't in the business of saving people's lives in EXACTLY this same way, on a regular basis, the estimate of 90% is probably crap. And so is the estimate of 100% probability of saving 400 lives. All you can really say, is that you see fewer difficulties that way, from where you are standing now. It's a crap shoot, either way, because, once you get started, no matter which option you choose, difficulties you hadn't anticipated will arise.

This reminds me of 'the bridge experiment', where a test subject is given the opportunity to throw a fat person off a bridge in front of a train, and thereby save the lives of 5 persons trapped on the tracks up ahead. The psychologists bemoaned the lack of rationality of the test subjects, since most of them wouldn't throw the fat person off the bridge, and thus trade the lives of one person, for five. I was like, 'ARE YOU CRAZY? Do you think one fat person would DERAIL A TRAIN? What do you think cow catchers are for, fool? What if he BOUNCED a couple of times, and didn't end up on the rails? It's preposterous. The odds are 1000 to 1 against success. No sane person would take that bet.'

The psychologists supposedly fixed this concern by telling the test subjects that it was guaranteed that throwing the fat person off the bridge would succeed. Didn't work, because people STILL wouldn't buy into their preposterous plan.

Then the psychologists changed the experiment so that the test subject would just have to throw a switch on the track which would divert the train from the track where the five people were trapped to a track where just one person was trapped (still fat by the way). Far more of the test subjects said they would flip the switch than had said they would throw someone off the bridge. The psychologists suggested some preposterous sounding reason for the difference, I don't even remember what, but it seemed to me that the change was because the plan just seemed a lot more likely to succeed. The test subjects DISCOUNTED the assurances of the psychologists that the 'throw someone off the bridge plan' would succeed. And quite rationally too, if you ask me. What rational person would rely on the opinion of a psychologist on such a matter?

When the 90%/500 or 100%/400 question was posed, I felt myself having exactly the same reaction. I immediately felt DUBIOUS that the odds were actually 90%. I immediately discounted the odds. By quite a bit, in fact. Perhaps that was because of lack of self confidence, or hard won pessimism from years of real life experience, but I immediately discounted the odds. I bet a lot of other people did too. And I wouldn't take the bet, for exactly that reason. I didn't BELIEVE the odds, as given. I was skeptical. Interestingly enough though, I was less skeptical of the 'can't fail/100%' estimate, than of the 90% estimate. Maybe I could easily imagine a scenario where there was no chance of failure at all, but couldn't easily imagine a scenario where the odds were, reliably, 90%. Once you start throwing around numbers like 90%, in an imperfect world, what you're really saying is 'there is SOME chance of failure'. Estimating how much chance, would be very much a judgement call.

So maybe what you're looking at here isn't irrationality, or the inability to multiply, but rather rational pessimism about it being as easy as claimed.

Well, it's perfectly obvious that as soon as convenient and cheap birth control becomes widely available, then many people will not have children. This reduces the birthrate. HOWEVER, the people that do have children will have been, on average, self-selected. And one thing you can be sure of, they will be over-represented in the psychological category of "People who really like kids"

That being the case, one has to assume that all our descendents, ultimately, will be extremely philoprogenic. Because all the people that deliberately don't have kids, because kids are 'more trouble than they are fun', will have eliminated themselves from the gene pool. In fact, they may eliminate themselves in a single generation.

Which says to me that we will have a bottleneck of sorts -- birth rates will go down, maybe a lot, and so will the total population, but then, as the philoprogenics begin to dominate the gene pool -- birth rates will start to climb again. And more so every generation, I expect. It's a positive feedback loop.

I've got to disagree with this one. Let's take a concrete example, say pity. The ability to feel pity is a complex adaptation, and so all persons feel pity. However HOW MUCH any one person feels pity for others is a highly variable quantity. It varies dramatically from person to person, and from situation to situation; moreover, some people, (eg psycopaths) don't feel any pity at all: their pity mechanism is broken, defective. Therefore the only conclusion you can draw from the alleged "Psychological unity of humankind" is that they will feel some unknown amount of pity in a certain situation, unless of course, they feel none at all.

The possible scale for pity ranges from 0 to some (unknown) maximum. Alleging the "psychological unity of humankind" gives you no additional information.

You can make the same null statement about every other human psychological trait, from conscientiousness to competitiveness.

You can't even state " 'Smorgraph' does not exist, since I have never felt it" because of course, your smorgraph generator or detector may be defective.

Lastly, since human beings were, until a couple of hundred years ago, in relatively isolated breeding pools, with fairly limited transfer of genetic materials between pools. This is still the case, although to a lesser degree. (compare the amount of gene transfer between Lima, Peru and Ontario, Canada vs the amount of gene transfer within the city of Toronto). It is highly likely that differential evolutionary pressure drove evolution in different direction in different sub-populations. The most famous examples of these are physical adaptations, like sickle cell anemia, an adaptation against malaria, and the ability to digest milk sugar, etc, but the same kind of evolutionary pressure no doubt also drove expression levels of psychological adaptations. Psychological mechanisms are obviously heritable and shaped by evolution. ie, in some ancestral environments, no doubt it was MORE ADVANTAGEOUS to be highly loyal, or to be very calm and dispassionate, or to be extremely vengeful and quick to anger, or to be deeply concerned about the well-being of your children, or to be very lustful, or extremely conscientious. These are adaptations, and your evolutionary environment is going to slide them up or down, based on differential survival rates.

Thus, it is HIGHLY PROBABLE that people in one ancestral sub-population are far more like EACH OTHER psychologically that they are psychologically like other human beings. The examples of this, which are legion, are commonly called 'prejudice'. Prior probability would probably be a more scientific term.

'The psychological unity of mankind' is nothing more than a fairy story, something good hearted (or weak headed) people WANT to believe, because they are afraid that people 'being different' will lead to pogroms and lynching and blind discrimination. Well, probably it does. But, it's still TRUE.