Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

War and/or Peace (2/8)

33 Post author: Eliezer_Yudkowsky 31 January 2009 08:42AM

(Part 2 of 8 in "Three Worlds Collide")

..."So the question then is - now what?"

The Lord Pilot jumped up, then, his face flushed.  "Put up shields.  Now.  We don't gain anything by leaving them down.  This is madness!"

"No," said the Ship's Confessor in professional tones, "not madness."

The Pilot slammed his fists on the table.  "We're all going to die!"

"They're not as technologically advanced as us," Akon said.  "Suppose the Babyeaters do decide that we need to be exterminated.  Suppose they open fire.  Suppose they kill us.  Suppose they follow the starline we opened and find the Huygens system.  Then what?"

The Master nodded.  "Even with surprise on their side... no.  They can't actually wipe out the human species.  Not unless they're a lot smarter than they seem to be, and it looks to me like, on average, they're actually a bit dumber than us."  The Master glanced at the Xenopsychologist, who waved her hand in a maybe-gesture.

"But if we leave the ship's shields down," Akon said, "we preserve whatever chance we have of a peaceful resolution to this."

"Peace," said the Lady Sensory, in a peculiar flat tone.

Akon looked at her.

"You want peace with the Babyeaters?"

"Of course -" said Akon, then stopped short.

The Lady Sensory looked around the table.  "And the Babyeater children?  What about them?"

The Master of Fandom spoke, his voice uncertain.  "You can't impose human standards on -"

With a blur of motion and a sharp crack, the Lady Sensory slapped him.

The Ship's Confessor grabbed her arm.  "No."

The Lady Sensory stared at the Ship's Confessor.

"No," the Confessor repeated.  "No violence.  Only argument.  Violence doesn't distinguish truth from falsehood, my Lady."

The Lady Sensory slowly lowered her hand, but not her eyes.

"But..." said the Master.  "But, my Lady, if they want to be eaten -"

"They don't," said the Xenopsychologist.  "Of course they don't.  They run from their parents when the terrible winnowing comes.  The Babyeater children aren't emotionally mature - I mean they don't have their adult emotional state yet.  Evolution would take care of anyone who wanted to get eaten.  And they're still learning, still making mistakes, so they don't yet have the instinct to exterminate violators of the group code.  It's a simpler time for them.  They play, they explore, they try out new ideas.  They're..." and the Xenopsychologist stopped.  "Damn," she said, and turned her head away from the table, covering her face with her hands.  "Excuse me."  Her voice was unsteady.  "They're a lot like human children, really."

"And if they were human children," said the Lady Sensory into the silence, "do you think that, just because the Babyeater species wanted to eat human children, that would make it right for them to do it?"

"No," said the Lord Pilot.

"Then what difference does it make?" said the Lady Sensory.

"No difference at all," said the Lord Pilot.

Akon looked back and forth between the two of them, and saw what was coming, and somehow couldn't speak.

"We have to save them," said the Lady Sensory.  "We have to stop this.  No matter what it takes.  We can't let this go on."

Couldn't say that one word -

The Lord Pilot nodded.  "Destroy their ship.  Preserve our advantage of surprise.  Go back, tell the world, create an overwhelming human army... and pour into the Babyeater starline network.  And rescue the children."

"No," Akon said.

No?

"I know," said the Lord Pilot.  "A lot of Babyeaters will die at first, but they're killing ten times more children than their whole adult population, every year -"

"And then what?" said the Master of Fandom.  "What happens when the children grow up?"

The Lord Pilot fell silent.

The Master of Fandom completed the question.  "Are you going to wipe out their whole race, because their existence is too horrible to be allowed to go on?  I read their stories, and I didn't understand them, but -"  The Master of Fandom swallowed.  "They're not... evil.  Don't you understand?  They're not.  Are you going to punish me, because I don't want to punish them?"

"We could..." said the Lord Pilot.  "Um.  We could modify their genes so that they only gave birth to a single child at a time."

"No," said the Xenopsychologist.  "They would grow up loathing themselves for being unable to eat babies.  Horrors in their own eyes.  It would be kinder just to kill them."

"Stop," said Akon.  His voice wasn't strong, wasn't loud, but everyone in the room looked at him.  "Stop.  We are not going to fire on their ship."

"Why not?" said the Lord Pilot.  "They -"

"They haven't raised shields," said Akon.

"Because they know it won't make a difference!" shouted the Pilot.

"They didn't fire on us!" shouted Akon.  Then he stopped, lowered his voice.  "They didn't fire on us.  Even after they knew that we didn't eat babies.  I am not going to fire on them.  I refuse to do it."

"You think they're innocent?" demanded the Lady Sensory.  "What if it was human children that were being eaten?"

Akon stared out a viewscreen, showing in subdued fires a computer-generated graphic of the nova debris.  He just felt exhausted, now.  "I never understood the Prisoner's Dilemma until this day.  Do you cooperate when you really do want the highest payoff?  When it doesn't even seem fair for both of you to cooperate?  When it seems right to defect even if the other player doesn't?  That's the payoff matrix of the true Prisoner's Dilemma.  But all the rest of the logic - everything about what happens if you both think that way, and both defect - is the same.  Do we want to live in a universe of cooperation or defection?"

"But -" said the Lord Pilot.

"They know," Akon said, "that they can't wipe us out.  And they can guess what we could do to them.  Their choice isn't to fire on us and try to invade afterward!  Their choice is to fire on us and run from this star system, hoping that no other ships follow.  It's their whole species at stake, against just this one ship.  And they still haven't fired."

"They won't fire on us," said the Xenopsychologist, "until they decide that we've defected from the norm.  It would go against their sense of... honor, I could call it, but it's much stronger than the human version -"

"No," Akon said.  "Not that much stronger."  He looked around, in the silence.  "The Babyeater society has been at peace for centuries.  So too with human society.  Do you want to fire the opening shot that brings war back into the universe?  Send us back to the darkness-before-dawn that we only know from reading history books, because the holos are too horrible to watch?  Are you really going to press the button, knowing that?"

The Lord Pilot took a deep breath.  "I will.  You will not remain commander of the Impossible, my lord, if the greater conference votes no confidence against you.  And they will, my lord, for the sake of the children."

"What," said the Master, "are you going to do with the children?"

"We, um, have to do something," said the Ship's Engineer, speaking up for the first time.  "I've been, um, looking into what Babyeater science knows about their brain mechanisms.  It's really quite fascinating, they mix electrical and mechanical interactions, not the same way our own brain pumps ions, but -"

"Get to the point," said Akon.  "Immediately."

"The children don't die right away," said the Engineer.  "The brain is this nugget of hard crystal, that's really resistant to, um, the digestive mechanisms, much more so than the rest of the body.  So the child's brain is in, um, probably quite a lot of pain, since the whole body has been amputated, and in a state of sensory deprivation, and then the processing slowly gets degraded, and I think the whole process gets completed about a month after -"

The Lady Sensory threw up.  A few seconds later, so did the Xenopsychologist and the Master.

"If human society permits this to go on," said the Lord Pilot, his voice very soft, "I will resign from human society, and I will have friends, and we will visit the Babyeater starline network with an army.  You'll have to kill me to stop me."

"And me," said the Lady Sensory through tears.

Akon rose from his chair, and leaned forward; a dominating move that he had learned in classrooms, very long ago when he was first studying to be an Administrator.  But most in humanity's promotion-conscious society would not risk direct defiance of an Administrator.  In a hundred years he'd never had his authority really tested, until now...  "I will not permit you to fire on the alien ship.  Humanity will not be first to defect in the Prisoner's Dilemma."

The Lord Pilot stood up, and Akon realized, with a sudden jolt, that the Pilot was four inches taller; the thought had never occurred to him before.  The Pilot didn't lean forward, not knowing the trick, or not caring.  The Pilot's eyes were narrow, surrounding facial muscles tensed and tight.

"Get out of my way," said the Lord Pilot.

Akon opened his mouth, but no words came out.

"It is time," said the Lord Pilot, "to see this calamity to its end."  Spoken in Archaic English: the words uttered by Thomas Clarkson in 1785, at the beginning of the end of slavery.  "I have set my will against this disaster; I will break it, or it will break me."  Ira Howard in 2014.  "I will not share my universe with this shadow," and that was the Lord Pilot, in an anger hotter than the nova's ashes.  "Help me if you will, or step aside if you lack decisiveness; but do not make yourself my obstacle, or I will burn you down, and any that stand with you -"

"HOLD."

Every head in the room jerked toward the source of the voice.  Akon had been an Administrator for a hundred years, and a Lord Administrator for twenty.  He had studied all the classic texts, and watched holos of famous crisis situations; nearly all the accumulated knowledge of the Administrative Field was at his beck and call; and he'd never dreamed that a word could be spoken with such absolute force.

The Ship's Confessor lowered his voice.  "My Lord Pilot.  I will not permit you to declare your crusade, when you have not said what you are crusading for.  It is not enough to say that you do not like the way things are.  You must say how you will change them, and to what.  You must think all the way to your end.  Will you wipe out the Babyeater race entirely?  Keep their remnants under human rule forever, in despair under our law?  You have not even faced your hard choices, only congratulated yourself on demanding that something be done.  I judge that a violation of sanity, my lord."

The Lord Pilot stood rigid.  "What -" his voice broke.  "What do you suggest we do?"

"Sit down," said the Ship's Confessor, "keep thinking.  My Lord Pilot, my Lady Sensory, you are premature.  It is too early for humanity to divide over this issue, when we have known about it for less than twenty-four hours.  Some rules do not change, whether it is money at stake, or the fate of an intelligent species.  We should only, at this stage, be discussing the issue in all its aspects, as thoroughly as possible; we should not even be placing solutions on the table, as yet, to polarize us into camps.  You know that, my lords, my ladies, and it does not change."

"And after that?" said the Master of Fandom suddenly.  "Then it's okay to split humanity?  You wouldn't object?"

The featureless blur concealed within the Confessor's Hood turned to face the Master, and spoke; and those present thought they heard a grim smile, in that voice.  "Oh," said the Confessor, "that would be interfering in politics.  I am charged with guarding sanity, not morality.  If you want to stay together, do not split.  If you want peace, do not start wars.  If you want to avoid genocide, do not wipe out an alien species.  But if these are not your highest values, then you may well end up sacrificing them.  What you are willing to trade off, may end up traded away - be you warned!  But if that is acceptable to you, then so be it.  The Order of Silent Confessors exists in the hope that, so long as humanity is sane, it can make choices in accordance with its true desires.  Thus there is our Order dedicated only to that, and sworn not to interfere in politics.  So you will spend more time discussing this scenario, my lords, my ladies, and only then generate solutions.  And then... you will decide."

"Excuse me," said the Lady Sensory.  The Lord Pilot made to speak, and Sensory raised her voice.  "Excuse me, my lords.  The alien ship has just sent us a new transmission.  Two megabytes of text."

"Translate and publish," ordered Akon.

They all glanced down and aside, waiting for the file to come up.

It began:

THE UTTERMOST ABYSS OF JUSTIFICATION
A HYMN OF LOGIC
PURE LIKE STONES AND SACRIFICE
FOR STRUGGLES OF THE YOUNG SLIDING DOWN YOUR THROAT-

Akon looked away, wincing.  He hadn't tried to read much of the alien corpus, and hadn't gotten the knack of reading the "translations" by that damned program.

"Would someone," Akon said, "please tell me - tell the conference - what this says?

There was a long, stretched moment of silence.

Then the Xenopsychologist made a muffled noise that could have been a bark of incredulity, or just a sad laugh.  "Stars beyond," said the Xenopsychologist, "they're trying to persuade us to eat our own children."

"Using," said the Lord Programmer, "what they assert to be arguments from universal principles, rather than appeals to mere instincts that might differ from star to star."

"Such as what, exactly?" said the Ship's Confessor.

Akon gave the Confessor an odd look, then quickly glanced away, lest the Confessor catch him at it.  No, the Confessor couldn't be carefully maintaining an open mind about that.  It was just curiosity over what particular failures of reasoning the aliens might exhibit.

"Let me search," said the Lord Programmer.  He was silent for a time.  "Ah, here's an example.  They point out that by producing many offspring, and winnowing among them, they apply greater selection pressures to their children than we do.  So if we started producing hundreds of babies per couple and then eating almost all of them - I do emphasize that this is their suggestion, not mine - evolution would proceed faster for us, and we would survive longer in the universe.  Evolution and survival are universals, so the argument should convince anyone."  He gave a sad chuckle.  "Anyone here feel convinced?"

"Out of curiosity," said the Lord Pilot, "have they ever tried to produce even more babies - say, thousands instead of hundreds - so they could speed up their evolution even more?"

"It ought to be easily within their current capabilities of bioengineering," said the Xenopsychologist, "and yet they haven't done it.  Still, I don't think we should make the suggestion.""

"Agreed," said Akon.

"But humanity uses gamete selection," said the Lady Sensory.  "We aren't evolving any slower.  If anything, choosing among millions of sperm and hundreds of eggs gives us much stronger selection pressures."

The Xenopsychologist furrowed her brow.  "I'm not sure we sent them that information in so many words... or they may have just not gotten that far into what we sent them..."

"Um, it wouldn't be trivial for them to understand," said the Ship's Engineer.  "They don't have separate DNA and proteins, just crystal patterns tiling themselves.  The two parents intertwine and stay that way for, um, days, nucleating portions of supercooled liquid from their own bodies to construct the babies.  The whole, um, baby, is constructed together by both parents.  They don't have separate gametes they could select on."

"But," said the Lady Sensory, "couldn't we maybe convince them, to work out some equivalent of gamete selection and try that instead -"

"My lady," said the Xenopsychologist.  Her voice, now, was somewhat exasperated.  "They aren't really doing this for the sake of evolution.  They were eating babies millions of years before they knew what evolution was."

"Huh, this is interesting," said the Lord Programmer.  "There's another section here where they construct their arguments using appeals to historical human authorities."

Akon raised his eyebrows.  "And who, exactly, do they quote in support?"

"Hold on," said the Lord Programmer.  "This has been run through the translator twice, English to Babyeater to English, so I need to write a program to retrieve the original text..."  He was silent a few moments.  "I see.  The argument starts by pointing out how eating your children is proof of sacrifice and loyalty to the tribe, then they quote human authorities on the virtue of sacrifice and loyalty.  And ancient environmentalist arguments about population control, plus... oh, dear.  I don't think they've realized that Adolf Hitler is a bad guy."

"They wouldn't," said the Xenopsychologist.  "Humans put Hitler in charge of a country, so we must have considered him a preeminent legalist of his age.  And it wouldn't occur to the Babyeaters that Adolf Hitler might be regarded by humans as a bad guy just because he turned segments of his society into lampshades - they have a custom against that nowadays, but they don't really see it as evil.  If Hitler thought that gays had defected against the norm, and tried to exterminate them, that looks to a Babyeater like an honest mistake -"  The Xenopsychologist looked around the table.  "All right, I'll stop there.  But the Babyeaters don't look back on their history and see obvious villains in positions of power - certainly not after the dawn of science.  Any politician who got to the point of being labeled "bad" would be killed and eaten.  The Babyeaters don't seem to have had humanity's coordination problems.  Or they're just more rational voters.  Take your pick."

Akon was resting his head in his hands.  "You know," Akon said, "I thought about composing a message like this to the Babyeaters.  It was a stupid thought, but I kept turning it over in my mind.  Trying to think about how I might persuade them that eating babies was... not a good thing."

The Xenopsychologist grimaced.  "The aliens seem to be even more given to rationalization than we are - which is maybe why their society isn't so rigid as to actually fall apart - but I don't think you could twist them far enough around to believe that eating babies was not a babyeating thing."

"And by the same token," Akon said, "I don't think they're particularly likely to persuade us that eating babies is good."  He sighed.  "Should we just mark the message as spam?"

"One of us should read it, at least," said the Ship's Confessor.  "They composed their argument honestly and in all good will.  Humanity also has epistemic standards of honor to uphold."

"Yes," said the Master.  "I don't quite understand the Babyeater standards of literature, my lord, but I can tell that this text conforms to their style of... not exactly poetry, but... they tried to make it aesthetic as well as persuasive."  The Master's eyes flickered, back and forth.  "I think they even made some parts constant in the total number of light pulses per argumentative unit, like human prosody, hoping that our translator would turn it into a human poem.  And... as near as I can judge such things, this took a lot of effort.  I wouldn't be surprised to find that everyone on that ship was staying up all night working on it."

"Babyeaters don't sleep," said the Engineer sotto vocce.

"Anyway," said the Master.  "If we don't fire on the alien ship - I mean, if this work is ever carried back to the Babyeater civilization - I suspect the aliens will consider this one of their great historical works of literature, like Hamlet or Fate/stay night -"

The Lady Sensory cleared her throat.  She was pale, and trembling.

With a sudden black premonition of doom like a training session in Unrestrained Pessimism, Akon guessed what she would say.

The Lady Sensory said, in an unsteady voice, "My lords, a third ship has jumped into this system.  Not Babyeater, not human."

To be continued...

Comments (61)

Sort By: Old
Comment author: Carl_Shulman 31 January 2009 09:06:29AM 1 point [-]

I wonder about the psychological mechanisms and intuitions at work in the Babyeaters. After all, human babies don't look like Babyeater babies, they're less intelligent, etc. Their intellectual extension of strong intuitions to exotic cases might well be much more flexible than their applications to situations from the EEA, e.g. satisfying them by drinking cocktails containing millions of blastocysts. Similarly, human intuitions start to go haywire in exotic sci-fi thought experiments and strange modern situations.

Comment author: Doug_S. 31 January 2009 10:03:02AM 1 point [-]

Are they referring to the anime or Visual Novel of Fate/stay night?

Comment author: Eliezer_Yudkowsky 31 January 2009 10:14:26AM 7 points [-]

Obviously the visual novel.

Comment author: Options 31 January 2009 10:26:40AM -2 points [-]

Any chance of getting an RSS or Atom feed for Overcoming Bias that has everyone but Eliezer's posts?

Comment author: Eliezer_Yudkowsky 31 January 2009 10:47:35AM 3 points [-]

Options, this will hopefully become easier shortly with Less Wrong. Sorry.

Comment author: adina 31 January 2009 12:30:26PM 7 points [-]

Options- I'm sure you're perfectly aware of many resources that could answer your question. Don't be passive-aggressive and pretend you simply want a question to be answered, when it is clear that you are just trying to be rude.

Comment author: simon2 31 January 2009 02:21:57PM 1 point [-]

Akon claims this is a "true" prisoner's dilemma situation, and then tries to add more values to one side of the scale. If he adds enough values to make cooperation higher value than defecting, then he was wrong to say it was a true prisoner's dilemma. But the story has made it clear that the aliens appear to be not smart enough to accurately anticipate human behaviour (or vice versa for that matter), so this is not a situation where it is rational to cooperate in a true prisoner's dilemma. If it really is a true prisoner's dilemma, they should just defect.

Of course, there may be a more humane approach than extermination or requiring them to live under human law: forcible modification to remove the desire to eat babies, and reduce the amount of reproduction. It might be a little tricky to do this without completely messing up the aliens' psychology.

Also, it seems a little unlikely that a third ship would arrive given that the arrival of even one alien ship was considered so surprising in the first installment.

Comment author: Martin4 31 January 2009 03:10:46PM 0 points [-]

> Also, it seems a little unlikely that a third ship would arrive given that the arrival of even one alien ship was considered so surprising in the first installment.

Don't you think there might be a good reason for that which we will learn next time? My guess would be so. that has way higher standards like 'killing the unfertilized egg is murder'.

Martin

Comment author: JamesAndrix 31 January 2009 03:25:14PM 3 points [-]

"Babyeaters don't sleep" could be a t-shirt.

Simon: I think you're missing the point, would you want to be modified to no longer value friendship? or to value the life of a roach more than friendship?

babyeaters evolved to not view their babies as something deserving of sympathy, and eating them is the most primal way they show social trustworthyness.

It's not just that it would mess up your psychology to change you this way, it would essentially murder you, and you would be fully opposed to it. (well, I think I would be.)

Comment author: Kaj_Sotala 31 January 2009 03:29:02PM 3 points [-]

I remain puzzled over why they're trying to decide on the correct course of action themselves.

Wouldn't the only reasonable decision in this case be to return to the rest of the humanity, let the actual government decide whether or not to go to war with an entirely new species? Sure, they'd lose the advantage of surprise, and it may be a really long way back home, but it still bothered me that not a single crew member even raised the possibility. If I was on the crew, pointing out that we have no right to make such a decision on our own would be the first thing I'd do.

Comment author: Kaj_Sotala 31 January 2009 03:35:15PM 2 points [-]

James Andrix: babyeaters evolved to not view their babies as something deserving of sympathy, and eating them is the most primal way they show social trustworthyness.

Oh, they did. That was pretty clearly stated in the first installment - they love their babies, and have great sympathy for them. That's what makes the inevitable winnowing so tragic, and that's why overcoming it - despite everything - is so heroic. If the winnowing wasn't any big deal, then obviously it wouldn't be in the center of their ethical system.

(Though I would imagine that they might care somewhat less for their children than humans - considering how eagerly they have developed to punish cheaters, one'd think there to be a considerable selection pressure operating in favor of those who didn't have problems with slaughtering their offspring. Of course, being entirely sosiopathic towards the kids would reduce the chances that even one of them would live on to reproduce.)

Comment author: spriteless 31 January 2009 03:52:16PM 1 point [-]

Hmm.

I'm already imagining three's sensor capabilities are advanced enough to see aliens, but they really don't like getting blown up. Upon seeing these two other aliens reach first contact and not blow each other up, it would be more reasonable to risk showing up.

Comment author: Vladimir_Nesov 31 January 2009 03:55:44PM 2 points [-]

After the last discussion of cooperation as meta-commitment, I found another intuition for cooperation in prisoner's dilemma: trade. If you are more efficient in doing things good for the other side than other side, and other side is more efficient in doing things for you than you are, then both of you should do things that other side likes, even if you don't. For example, if it turns out that humanity is sufficiently better positioned for making paperclips than paperclip maximizer, and paperclip maximizer is better positioned to create a Friendly future than we are, then cooperation on our side can consist in us killing off ourselves and starting to make paperclips, and paperclipper should stop making paperclips and become Friendly. Essentially, you are physically switching places with an enemy, and as a result you both a better off. Your mind starts to host enemy's mind with enemy's morality, and enemy's mind start to host your mind with your morality. This switching can be temporary as well as permanent, and sides in this trade can be located anywhere in time or space. The only prerequisite is that you both know what the other side wants, and how things are expected to turn out given your actions, even if you are unable to communicate, ever.

Comment author: Chris_Yeh 31 January 2009 03:57:45PM 2 points [-]

Methinks Elie is making to too easy on his human characters. I actually don't feel much emotional angst over the babyeaters because they are so alien. After all, plenty of species on Earth practice cannibalism, yet we don't go on a crusade to exterminate them.

No, what he really needs is an alien race that consists of cuddly mammals, or perhaps an offshoot of humanity that evolved this practice of spawning and culling.

Most people wouldn't feel horror over crystalline entities eating their young, but they would go apeshit over human beings doing the same.

Comment author: Martin4 31 January 2009 04:09:31PM 1 point [-]

> After all, plenty of species on Earth practice cannibalism, yet we don't go on a crusade to exterminate them.

Sentience DOES make a difference. You dont frown on your cat for hunting mice, but on your dog for doing it with children.

Martin

Comment author: Cabalamat2 31 January 2009 04:12:37PM 1 point [-]

I don't know what political setup the humans have, but it probably doesn't extent to Akon and his crew choosing war for the whole human species. Wouldn't the wise thing to do be to report back, especially considering they have some very important news?

Comment author: Manon_de_Gaillande 31 January 2009 04:22:06PM -2 points [-]

Eliezer, why do you hate death so much? I understand why you'd hate it as much as the social norm wants you to say you do, but not so much more. People don't hate death, and don't even say they hate death nearly as much as you do. I can't think of a simpler hypothesis than "Eliezer is a mutant".

Now, of course, throwing in the long, painful agony of children changes something.

Comment author: Tamfang 13 August 2010 07:03:56AM *  5 points [-]

"Every human culture had expended vast amounts of intellectual effort on the problem of coming to terms with death. Most religions had constructed elaborate lies about it, making it out to be something other than it was – though a few were dishonest about life, instead. But even most secular philosophies were warped by the need to pretend that <i>death was for the best</i>. ¶ It was the naturalistic fallacy at its most extreme — and its most transparent, but that didn't stop anyone. Since any child could tell you that death was meaningless, contingent, unjust, and abhorrent beyond words, it was a hallmark of sophistication to believe otherwise." —Margit in '<a href="http://www.gregegan.net/BORDER/Border.html">Border Guards</a>' by Greg Egan

Comment author: Gray_Area 31 January 2009 04:39:27PM 4 points [-]

For what it's worth, I find plenty to disagree with Eleazar about, on points of both style and substance, but on death I think he has it exactly right. Death is a really bad thing, and while humans have diverse psychological adaptations for dealing with death, it seems the burden of proof is on people who do NOT want to make the really bad thing go away in the most expedient way possible.

Comment author: Daniel_Armak 31 January 2009 05:09:24PM 2 points [-]

> Sentience DOES make a difference. You dont frown on your cat for hunting mice, but on your dog for doing it with children.

That's at least partly due to speciesm. How many people have gone on crusades to stop leopards from eating chimpanzees? For that matter, how many people devote their lives to stopping other *humans* from eating chimpanzees?

As for cannibalism, it seems to me that its role in Eliezer's story is to trigger a purely illogical revulsion in the humans who antropomorphise the aliens.

Imagine two completely different alien species living in one (technological) society, where each eats and "winnows" the other's children. This is the natural, evolved behavior of both species, just as big cats eat apes and (human) apes eat antelopes.

No cannibalism takes place, but the same amount of death and suffering is present as in Eliezer's scenario. Should we be less or more revolted at this? Which scenario has the greater moral weight? Should we say the two-species configuration is morally superior because they've developed a peaceful, stable society with two intelligent species coexisting instead of warring and hunting each other?

Comment author: Daniel_Armak 31 January 2009 05:23:13PM 0 points [-]

Also, if some people care so much about this crusade they're willing to go against the rest of human society and risk a huge war, then logically they ought to have mounted a huge operation long ago to sweep the galaxy looking for morally unsuitable aliens. Killing or forcefully transforming any alien species that 1) they judge to be sufficiently intelligent and 2) whose behavior doesn't conform to human morals.

Or they might realize there's no real upper bound on the amount of suffering that might potentially be taking place somewhere out of sight. Especially if you give more weight to the suffering or death of more intelligent individuals. In which case they might want to make an alliance with the Baby Eaters to search the galaxy for cultures so alien that they would be abominations to both species. And only exterminate the Baby Eaters once the galaxy has been swept clean.

Put like that, it seems to me to be a really bad idea. But isn't that what follows from the Pilot's argument? If stopping the Baby Eating is so important they're willing to risk the extermination of humanity for it. (And there's no way they could be sure of the Baby Eaters' potential in a species-wide war just from reading one badly translated and possibly censored alien library for a day. So they're proposing going to war where they can't be sure of victory.)

Comment author: Kaj_Sotala 31 January 2009 05:54:25PM 2 points [-]

Daniel Armak: But isn't that what follows from the Pilot's argument?

I don't think the Pilot is really taking the time to think through all the logical consequences of what he's saying, to be honest.

Comment author: Tim_Tyler 31 January 2009 06:01:29PM 1 point [-]

I note that filial cannibalism is quite common on this planet.

Gamete selection has quite a few problems. It only operates on half the genome at a time - and selection is performed before many of the genes can be expressed. Of course gamete selection is cheap.

What spiders do - i.e. produce lots of offspring, and have many die as infants - has a huge number of evolutionary benefits. The lost babies do not cost very much, and the value of the selection that acts on them is great.

Human beings can't get easily get there - since they currently rely on gestation inside a human female body for nine months, but - make no mistake - if we could produce lots of young, and kill most of them at a young age, then that would be a vastly superior system in terms of the quantity and quality of the resulting selection.

Human females do abort quite a few foetuses after a month or so - ones that fail internal and maternal integrity tests - but the whole system is obviously appalingly inefficient.

Comment author: Daniel_Armak 31 January 2009 06:18:16PM 3 points [-]

> I don't think the Pilot is really taking the time to think through all the logical consequences of what he's saying

Indeed, even if he wants to make war, the logical next step would still be to keep talking to the aliens and learning as much as possible about them. Then maybe trying to capture or infiltrate their ship. Or asking for escort to their system and returning with strategic knowledge about that. Preparing a surprise attack. Things like that.

Destroying the first contact alien ship would be stupid.

Comment author: Jotaf 31 January 2009 07:10:50PM 1 point [-]

I guess the ship's "council" making the decisions helps the argument that Eliezer is making, and can be waived simply because you *could* have a lengthier story where they traveled back to Earth and *then* the Earth Government had exactly the same debate. But that's nitpicky, how would it help the story or the argument behind it? IMHO it's good the way it is.

Some of the council's members having these extreme reactions of empathy seems a bit alien to us even, but that is our own bias. We ignore suffering in so-called 3rd world countries every day. "Aliens eating each other? Please! Live and let die." Who's right on this one, us or them?

The council members live in a world where humans (the only sentients they know) don't fight other humans, and that's it. Suffering is considered the most amoral thing by their standards. I think it's entirely in-character and reasonable that they feel compelled to stop it.

Comment author: tim4 31 January 2009 07:20:16PM 2 points [-]

To those wondering why the crew doesn't report back: isn't it even more implausible that two alien civilizations exchange petabytes of information about themselves, translate each other's languages, and start a philosophical discussion within hours of contacting each other, as opposed to, oh I don't know, almost anything else? This is a stage for some archetypes to discuss some ethical point, and you can assume that they will think of everything that Eliezer wants to cover (or else appropriate actors will appear at the right time). That's all that matters.

Implausibilities abound, but whether or not they matter depends on one's assumptions about the goals of the story. Given that I'm not yet sure what Eliezer's goals are, I come up with my own. (Actually, I would do this anyway). I can't help but wonder what it would be like to be a Babyeater, and I ask all sorts of questions that may be completely irrelevant to Eliezer's purpose. For example: How could they not be able to distinguish between the concept of good and the concept of baby eating if they understand that survival is good and that they survived before agriculture and before they started eating babies? When are babies taught that baby eating is synonymous with good? Do babies realize what will happen to them? We know that they do not want to be eaten, so do any of them try to make reasoned arguments about why eating them is unnecessary? Have any of them tried to organize the babies in revolt? I understand that they are young and so presumably not very capable, but still. How are the survivors selected? Does the evolutionary process the Babyeaters like so much optimize for Babyeaters who run fast, or are promising babies selected early and separated from the rest of the group? If the latter, what are the criteria? How do the dynamics of Babyeater society change with their ability to detect cheaters? What are their reasons for thinking that they are good at it? Etc. etc. Are any of these questions relevant? It depends.

The point is not that Babyeaters are implausible, it's that Babyeaters are fascinating enough to think about in some detail, that doing so will eventually raise questions of plausibility, and that such questions are likely to be unconstrained by their relevance the the author's point if 1) it's not yet obvious what the author's point is, and/or 2) they author's point is, or becomes, relatively less interesting than questions about Babyeater society and evolution. After all, p > 0 where p is the probability that Eliezer's real goal is to make OvercomingBias the number one search result for "baby eating." (The previous sentence is a joke.)

TL;DR: Properly applying the MST3K Mantra is non-trivial.

Comment author: Daniel_Armak 31 January 2009 07:36:40PM 0 points [-]

> Do babies realize what will happen to them?

I asked about this in yesterday's comments thread, but I guess everyone's moved here since then :-)

My intuition is that selection pressure on young aliens (to do anything it takes not get eaten) would be stronger than most selection pressure adults experience (most adults produce hundreds of offspring <=> only one offspring out of several hundred survives; and in a technological society most if not all adults live to reproduce).

We should see children evolving to escape being eaten. If running faster doesn't work, then by hurting other children to make them run slower. Or by children eating one another themselves. Or by a social organization that lets a few bullies/rulers/... send other children to be eaten in their stead. Or by evolving to be poisonous or at least tasting really bad and having orange-black striping to warn your parents :-)

Also, the period of time from birth to the beginning of (post-winnowing) growth spurt would be compressed to the utter minimum required by their physiology. (The faster you grow up, the smaller the window of danger to be eaten). On that basis, the pre-winnowing children may not have much time to philosophize about being eaten.

Eventually you get is a creature that's born sentient, manages to learn about the day/night cycle (plus whatever inborn "instinct" provides), and then the winnowing takes place at the age of 2 days before the growth spurt can begin. Very stylized, kind of thing.

Comment author: Thom_Blake 31 January 2009 08:05:41PM 0 points [-]

spriteless,

That's what I was thinking. Perhaps the newcomer engineered this meetup somehow to see whether the two species are safe to contact.

Comment author: Marcello 31 January 2009 08:18:19PM 0 points [-]

""" "Out of curiosity," said the Lord Pilot, "have they ever tried to produce even more babies - say, thousands instead of hundreds - so they could speed up their evolution even more?"

"It ought to be easily within their current capabilities of bioengineering," said the Xenopsychologist, "and yet they haven't done it. Still, I don't think we should make the suggestion.""

"Agreed," said Akon. """

That's not the least bit obvious. Do we really want the Babyeaters to hold back corresponding suggestions that might make our society better from our perspective and worse from theirs?

If, in this situation, we ought to bite the prisoner's-dilemma bullet to the degree of not invading the Babyeater planet because peaceful situations are, on average, better than war-torn situations, doesn't the same argument mean that we shouldn't hold back helpful advice, provided that, on the whole, situations in which helpful advice is given freely are better?

Now maybe it's the case that if we swapped that particular kind of helpful advice with the baby eaters, the degree to which Babyeater planet got worse by our standards is more than the degree to which our planet would bet better by our standards, and vise versa. But in that case it would be better for both sides to draw up a treaty....

Comment author: Kevin7 31 January 2009 08:19:48PM 0 points [-]

Manon: I hesitate to post this because it's such an emotional piece, but Eliezer has already written an impassioned response to your question. http://yudkowsky.net/other/yehuda

Comment author: Larry_D'Anna 31 January 2009 08:33:56PM 0 points [-]

Did Ira Howard actually say that? In which story?

Comment author: Tamfang 13 August 2010 07:01:59AM 0 points [-]

That Ira Howard died well before 1911, let alone 2014.

Comment author: Furcas 31 January 2009 09:34:08PM 1 point [-]

Brilliant, Eliezer. I love the concept of the Order of Silent Confessors. It makes the distinction between terminal values and the conduct that one should adopt to uphold those values crystal clear. That said, the thought of an organization of people who are willing to, um, purge themselves of all their terminal values except one (to help humans fulfill their fundamental desires, whatever they may be) is a bit hard to believe.

""Out of curiosity," said the Lord Pilot, "have they ever tried to produce even more babies - say, thousands instead of hundreds - so they could speed up their evolution even more?"

"It ought to be easily within their current capabilities of bioengineering," said the Xenopsychologist, "and yet they haven't done it.""

Isn't this evidence that baby eating is not, in fact, one of the Babyeaters' terminal values? If it really was they would do everything to increase the amount of babies they eat. It looks like their terminal value, instead of being "eating babies", is a actually something like, "eating babies in the way that our ancestors have always eaten babies". In other words, they put more value on upholding the _tradition_ of baby eating than on baby eating as such.

Comment author: Fetterkey 23 July 2009 08:06:46AM 1 point [-]

This is extremely belated, but I know several people who would be willing to eliminate the vast majority of their values in this fashion, at least if they believed that they were truly helping humanity.

Comment author: simon2 31 January 2009 10:57:05PM 0 points [-]

James Andrix: I don't claim that the aliens would prefer modification over death, only that it is more consonant with my conception of human values to modify them than exterminate them, notwithstanding that the aliens may prefer the latter.

Comment author: Vladimir_Nesov 01 February 2009 12:13:07AM 0 points [-]

@Marcello: Every decision to give a gift on your side corresponds to a decision to abstain from accepting your gift on the other side. Thus, decisions to give must be made on case-to-case basis, cooperation in true prisoner's dilemma doesn't mean unconditional charity.

Comment author: Kevin7 01 February 2009 01:47:29AM 0 points [-]

This story is so Asimovian that I keep waiting for "violence is the last refuge of the incompetent" to pop up.

Comment author: Kevin7 01 February 2009 01:48:13AM 0 points [-]

And is Fate/Stay night really that good?

Comment author: Marcello 01 February 2009 01:52:04AM 0 points [-]

Vladimir says: """Every decision to give a gift on your side corresponds to a decision to abstain from accepting your gift on the other side. Thus, decisions to give must be made on case-to-case basis, cooperation in true prisoner's dilemma doesn't mean unconditional charity."""

Agreed. Obviously (for example) the human ship shouldn't self-destruct. But I wasn't talking about all gifts, I was talking about the specific class of gifts called "helpful advice." And I did specify: "provided that, on the whole, situations in which helpful advice is given freely are better."

I was comparing the two strategies "Don't give away any helpful advice of the level the other party is likely to be able to reciprocate" and "give away all helpful advice of the level the other party is likely to be able to reciprocate" and pointing out that maybe they form another prisoner's dilemma. Of course, there may be more fine-grained strategies that work even better, strategies that actually take into account the relative amount of good and bad each piece of advice brings to the two parties. But remember that you must also consider how your strategy is going to be chronophoned over to the baby eaters. If we make the first gift, what exchange rate of baby-eater utilons for human utilons do we tolerate? (If the gifts are made of information, it may be impossible for trades to be authenticated without the possibility of other party taking the gift and using it (though of course it might be that the equilibrium has an honor system....)) It looks like it gets really complicated. Worth thinking about? Yes, but right now I'm busy.

Comment author: Eliezer_Yudkowsky 01 February 2009 01:59:56AM 2 points [-]

Simon: Also, it seems a little unlikely that a third ship would arrive given that the arrival of even one alien ship was considered so surprising in the first installment.

There are lots of starlines leading out from each system. They're somewhat expensive to open initially, then stay open. The nova acted as a rendezvous signal, causing all starlines leading to that star to fluctuate. Humans and aliens had never before explored the same world, but in this case, three different alien species had explored a world with a starline to the nova system. Without the nova, they never would have found one another.

Chris Yeh: Most people wouldn't feel horror over crystalline entities eating their young, but they would go apeshit over human beings doing the same.

Suppose I put your identical mind (including all memories, unchanged) into a crystalline body. Would you stop empathizing with yourself? How much do I have to change a human child's body (leaving the brain the same) before you would stop caring if they got eaten? How about a child severely disfigured by burns - do you stop empathizing with them once they no longer have a human-looking face and skin?

Kaj Sotala: Wouldn't the only reasonable decision in this case be to return to the rest of the humanity, let the actual government decide whether or not to go to war with an entirely new species? Sure, they'd lose the advantage of surprise

They're not going to duck out on the responsibility if that means already making the decision, e.g., losing the advantage of surprise. They have to decide now whether to fire on the Babyeater ship.

Armak: No cannibalism takes place, but the same amount of death and suffering is present as in Eliezer's scenario. Should we be less or more revolted at this?

Exactly as revolted. The problem isn't cannibalism, it's children being eaten.

Indeed, even if he wants to make war, the logical next step would still be to keep talking to the aliens and learning as much as possible about them.

The Babyeaters at least seem to have dumped their local Net, which removes some of that incentive, and the course of action you suggest is not without risk.

tim: How could they not be able to distinguish between the concept of good and the concept of baby eating if they understand that survival is good

What good is life without eating babies? How can you not understand that tribal loyalty is good?

Larry D'Anna: Did Ira Howard actually say that? In which story?

He didn't.

Furcas: It looks like their terminal value, instead of being "eating babies", is a actually something like, "eating babies in the way that our ancestors have always eaten babies". In other words, they put more value on upholding the _tradition_ of baby eating than on baby eating as such.

Clearly you don't value sex with your lover, since you're not having sex with him/her every minute of every day; you put more value on upholding the tradition of sex, rather than sex as such.

Comment author: Tamfang 13 August 2010 06:59:57AM *  2 points [-]

<i>Clearly you don't value sex with your lover</i>

Be serious: does anyone value fucking <em>as a terminal value</em> rather than as a means to enjoyment?

(assuming I understand the phrase 'terminal value', which is new to me)

Comment author: Daniel_Franke 01 February 2009 02:18:00AM 0 points [-]

Kevin, seconded. I'm half-expecting Eliezer to copy-paste a few paragraphs from the climax of Foundation's Edge into the middle of the story in order to see if anyone notices :-)

Comment author: Nick_Tarleton 01 February 2009 02:28:15AM 0 points [-]

tim: How could they not be able to distinguish between the concept of good and the concept of baby eating if they understand that survival is good

What good is life without eating babies? How can you not understand that tribal loyalty is good?

I'm afraid I don't understand. By "not a babyeating thing", do you mean that the Babyeaters (unlike us) use the same term/concept for the 'separable essence of goodness' as they do for one particular terminal value? Or do you just mean that convincing them not to eat babies would be analogous, and analogously hard, to convincing humans to eat babies?

Oh, yeah, great story.

Comment author: Furcas 01 February 2009 02:41:23AM 1 point [-]

Eliezer wrote: "Clearly you don't value sex with your lover, since you're not having sex with him/her every minute of every day; you put more value on upholding the tradition of sex, rather than sex as such."

I value sex with my girlfriend, but I also value lots of other things with comparable or greater intensity; these other desires are therefore in competition for my time and attention with my desire to have sex; as a result, I spend _some_ time having sex, and some time reading posts on Overcoming Bias. I don't spend all my time having sex because I don't value sex _that_ much.

However, if I thought that having sex was the best and most moral thing I could possibly do with my time, I would do everything in my power to have as much sex as possible. If I spent any time working, it would be with the goal of earning enough money to pay for food, lodging, etc, in order to be able to have sex later.

And if someone offered to alter my genes to, say, do away with my refractory period, so that I could screw my girlfriend nonstop, I'd agree in a nanosecond.

Comment author: Vladimir_Nesov 01 February 2009 03:45:58AM 0 points [-]

@Marcello: I assumed you agree that increasing the babyeating problem tenfold isn't something you'd expect to be reciprocated, not without knowing something they presently don't, and so the issue actually should be dismissed on that ground for the time being. It seems that you didn't start from this premise. Where you expect to profit -- sure, it's normal trade at that point.

The trick with cooperating in prisoner's dilemma is primarily in decision-theoretic setting, where you've only got one decision that's estimated over everything. The thesis is that cooperation is not what you get as instrumental strategy from structure of a game, it's what you start from as terminal choice (and can lose in structure of the game). It doesn't translate well to bounded rationality, sometimes you have to do what looks like defecting because you don't know the consequences.

For example, cooperation result should extend to a setting where one player observes the decision of the other player. Should I cooperate, knowing that the other player will observe my decision before making his? It looks like I shouldn't, unless I have a way of knowing that he cooperates, just expecting him to do that in order to be in the position to receive my cooperation doesn't work (unless he really makes a commitment/changes his utility, and presents evidence). But if I have the predictive power of Omega, sure, cooperation as the right decision in that setting is what I'd expect.

Comment author: Gwern_Branwen 01 February 2009 04:22:03AM 2 points [-]

Kevin: I don't think Eliezer meant to seriously suggest FSN is as good as Hamlet, but rather to continue his theme of 'strange future' (and maybe as part of a background viewpoint that 'one period's high culture is a former period's low pop culture' - which is true of Shakespeare BTW).

That said, I've always felt based on the animes that Tsukihime was Type-Moon's best work, and not FSN.

Comment author: Marcello 01 February 2009 05:16:21AM 1 point [-]

Vladimir says: "I assumed you agree that increasing the babyeating problem tenfold isn't something you'd expect to be reciprocated"

Aye, not necessarily. But perhaps the gesture of good will might be large enough to get the babyeaters to, say, take a medicine which melts the brains of their children right after they're eaten. They might be against such a medicine, but since they didn't evolve *knowing* that their babies were being slow-tortured for a month, they might not have desires against the medicine stronger than the desires in favor of having ten times as many kids. (And because the humans have tech. superiority, they could actually enforce the deal if that's necessary.)

It's a tricky ethical question knowing whether the humans are better off with that deal. And it's a tricky question of baby-crunch-crunch whether the baby-eaters are more-baby-eaten with that deal. But maybe there are better deals than the one I was able to think of in ten minutes.

Comment author: Marcello 01 February 2009 05:21:35AM 0 points [-]

Vladimir says: "I assumed you agree that increasing the babyeating problem tenfold isn't something you'd expect to be reciprocated"

Aye, not necessarily. But perhaps the gesture of good will might be large enough to get the babyeaters to, say, take a medicine which melts the brains of their children right after they're eaten. They might be against such a medicine, but since they didn't evolve *knowing* that their babies were being slow-tortured for a month, they might not have desires against the medicine stronger than the desires in favor of having ten times as many kids. (And because the humans have tech. superiority, they could actually enforce the deal if that's necessary.)

It's a tricky ethical question knowing whether the humans are better off with that deal. And it's a tricky question of baby-crunch-crunch whether the baby-eaters are more-baby-eaten with that deal. But maybe there are better deals than the one I was able to think of in ten minutes.

Comment author: Doug_S. 01 February 2009 07:29:11AM 1 point [-]

I am somewhat more disturbed by the suffering of the eaten babies than by the baby-eating itself. I don't like the baby-eating but I could tolerate it by chalking it up to Bizarre Alien Biology or whatever, but it should be possible to euthanize the babies before they are eaten, or whatever. Basically, I hate pain more than I hate death.

Consider the typical human reaction to the treatment of food animals in factory farms...

Comment author: Eliezer_Yudkowsky 01 February 2009 07:54:03AM 9 points [-]

So I was like "Here's my dystopian story of a world where a poorly programmed AI separates men and women onto different planets" and people were like "That's not a dystopia, I would totally live there" and I was like "You're just being contrarian" and they were like "No we're not" so then I was like "Okay here's my story about aliens who eat children" and they were like "We're cool with that" and I was like "..."

Comment author: James_Blair 01 February 2009 08:08:55AM 0 points [-]

I'm not cool with it but I just can't connect with it.

That said, maybe the Babyeaters need to eat the human's children to show them how Good it really is. If that fails to convince them, it's clobbering time.

Comment author: Nick_Tarleton 01 February 2009 09:11:04AM 0 points [-]

Never mind my previous comment. Obviously I cannot read.

Comment author: Tamfang 13 August 2010 07:05:58AM 0 points [-]

<i>"The Babyeaters don't seem to have had humanity's coordination problems. Or they're just more rational voters. Take your pick."</i>

My pick is that they rationalize like the dickens.

Comment author: sdmitch16 12 December 2011 12:16:53PM -2 points [-]

It was said that the aliens would not accept gamete reproduction because they ate children before they knew what evolution was. They did it because it increased the number of surviving children from each pen/ couple. If they are still culling to increase the success rate, than a new method that increased their success rate would be preferable. In the story, gametes are better for evolution than culling so gametes would produce more surviving off-spring and would thus be preferable to culling. I said "in the story" gametes are better because I disagree with the author on which is better for evolution. Producing good gametes will allow one to get an off-spring instead of rival mates, but the best gamete doesn't necessarily make the best off-spring. Gametes are selected by which one is the best swimmer and which has the best 'sense of smell'. It could be coded to produce a child with extremely low intelligence or extremely poor muscle tone, or unable to get glycogen out of their cells (Pompe). If out of the millions of sperm emitted with each ejaculation, the best tended to win, there would be much lower rates of genetic disease. They said that the alien would expect that if Hitler were bad he wouldn't have been made a leader or would have been killed. Most of the world disliked genocide, but the Nazi, being loyal, liked what ever Germany was doing so they liked genocide. This group was wrong and is the reason Hitler was the leader. I'm not saying the author messed up due to this ; its just something I noticed. A potential flaw was it was said if Hitler had been bad humans would have killed him. We did try to kill him and then tried the people who protected him for war crimes, although the Nazis being a group, it might have been seen as a 'reasonable error" esp. since most Nazis weren't killed.

Comment author: MixedNuts 12 December 2011 12:25:46PM 4 points [-]

See, that's why you don't say "because" about evolution. They eat children because-as-cognitive-motivation they have a drive to eat children. They have a drive to eat children because-as-evolutionary-history it increased the number of surviving children.

Comment author: Vladimir_Nesov 12 December 2011 01:31:59PM 0 points [-]

In addition to what MixedNuts said, see this post and its dependencies.

Comment author: Catnip 16 December 2011 07:58:22PM *  0 points [-]

Some observations:

  • Confessors seem to be a future form of beisutsukai.
  • Spam still exists or is still widely remembered in the distant future.
  • Even aliens recognize Hitler as central figure of human ethics. (I think I already encountered this statement on LessWrong, but can't quite remember where...)
Comment author: [deleted] 02 November 2013 11:15:21PM 2 points [-]

sotto vocce

voce

Comment author: wobster109 13 March 2014 03:17:12AM 1 point [-]

I would love to read that Babyeater poem.

Comment author: topynate 21 March 2014 11:13:28PM 1 point [-]

I want a training session in Unrestrained Pessimism.

Comment author: wafflepudding 24 May 2016 02:08:20AM 0 points [-]

Hmm… does some instance of utility get multiplied by the number of people who find it utilitous? Like, if there are twice as many humans, does that mean that one Babyeater baby eaten subtracts twice as much from group utility?