The Trolley Problem in popular culture: Torchwood Series 3

16 Post author: botogol 27 July 2009 10:46PM

It's just possible that some lesswrong readers may be unfamiliar with Torchwood: It's a British sci-fi TV series, a spin-off from the more famous, and very long-running cult show Dr Who.

Two weeks ago Torchwood Series 3 aired. It took the form of a single story arc, over five days, shown in five parts on consecutive nights. What hopefully makes it interesting to rationalist lesswrong readers who are not (yet) Whovians was not only the space monsters (1) but also the show's determined and methodical exploration of an iterated Trolley Problem:  in a process familiar to seasoned thought-experimenters the characters were tested with a dilemma followed by a succession of variations of increasing complexity, with their choices ascertained and the implications discussed and reckoned with.

An hypothetical, iterated rationalist dilemma... with space monsters... and monsters a great deal more scary - and messier - than Omega -  what's not to like?

So, on the off chance that you missed it, and as a summer diversion from more academic lesswrong fare, I thought a brief description of how a familiar dilemma was handled on popular British TV this month, might be of passing interest (warning: spoilers follow)

The details of the scenario need not concern us too much here (and readers are warned not too expend too much mental energy exploring the various implausibilities, for want of distraction) but suffice to say that the 456, a race of evil aliens, landed on Earth and demanded that a certain number of children be turned over to them to suffer a horrible fate-worse-than-death or else we face the familiar prospects of all out attack and the likely destruction of mankind.

Resistance, it almost goes without saying, was futile.

The problems faced by the team could be roughly sorted into some themes

The Numbers dilemma - is it worth sacrificing any amount of children to save the rest?

  • first (2) the 456 demand just 35 children
  • but later they demand 10% of the child population.

The Quality dilemma: does it make any difference which children?

  • what if, say, the sacrificed children are solely those whom nobody will miss?

The choice dilemma: how should the sacrifical cohort be chosen?

  • are there any criteria that are fairer, or more rational, than the obvious lottery?
  • other than 'fairness' what other criteria can you think of for selecting the victims?

The limits of human rationality: are there certain 'rational' decisions that are simply too much to expect a human being to be able to make?

  • what if it is your own children? Grand-children? Nephews? Neighbours? 

Actually despite my jocular tone in the first paragraph I don't want to make too light of this series, as it was disturbing viewing.

Anyway: that being said: rationalist lesswrong community members may want to think dispassionately about the their answers before I reveal the conclusions that Russell T Davies (the writer) came to:

Numbers

  • Lesswrongers will be encouraged to learn that the Torchwood characters were rationalists to a man and woman - there was little hesitation in agreeing to the 456's demands.
  • A rational attempt to negotiate (600 children were offered) was faced down by the 465's intemperate release of swine flu into the negotiating chamber.

Quality

  • In the Torchwood universe it made a great deal of difference which children - or units as they were plausibly referred to - were sacrificed, with governments paying attention to round up the orphans, the refugees and the unloved - for the unexpectedly rational reason of minimising the suffering of the survivors.

Choice

This was handled by the politicians who considered two dimensions in the selection:

  •  
    • the Darwinian dimension: could they improve the human gene pool by sacrificing the least desirable 10%?  If this seemed like a good rationalist plan (it did) then how could they quickly identify the losers? Answer: in a UK-centric political twist they chose those attending the schools with the poorest exam results.
    • the question of practicality and enforcabilty. This was the consideration that caused the children of decision-makers and enforcers to be spared. In a cute nested trolley problem the children of the soldiers were saved in order to ensure the cooperation of their parents in rounding up the unlucky and underserving, in order to ensure the survival of the rest.
    • at no stage was it necessary to push a fat man off a bridge

Rationality at the limit

On the question of 'how close' a straightforward evolutionary approach was used. Children of the decision-makers were safe, and grandchildren.

"And our nephews?" "Don't push it".

But the limits of rationality, it seems, are dependent upon gender: While it was recognised that no woman could be expected to agree to the rational sacrifice of her child, it was expected by some that men might have to, and in the end the main character - male - sacrificed a grandchild.

And that's it. Perhaps not a complete disposal of the trolley problem, but nevertheless an interesting excursion into the realms of philosphical dilemmas for a popular drama.   Rationalism is a meme - pass it on.


(1) like many TV aliens: surprisingly able to construct spaceships without the benefit of an opposable thumb
(2) Yes, that was actually the 1965 back-story.

Comments (86)

Comment author: ShardPhoenix 28 July 2009 10:41:52AM *  10 points [-]

It annoys me how people on TV or in movies who have to make tough, unpopular decisions are almost never shown to be right - there always turns out to be a deus ex machina or other third way out. From a spoiler given in another comment here it seems like this is yet another case of that - not that that's surprising when RTD is writing.

Comment author: jwdink 29 July 2009 02:59:31AM 3 points [-]

A good example of this (I think) is The Dark Knight, with the two ferries.

Comment author: michaelkeenan 29 July 2009 06:21:38AM 1 point [-]

Agreed. The one that annoys me the most is in the first Spiderman movie (spoiler warning) when the Green Goblin drops Mary-Jane and a tram full of child hostages, forcing Spiderman to choose who to save. I was excited to see what his choice would be...but then he just saves everyone.

Comment author: wedrifid 29 July 2009 08:08:56AM 0 points [-]

And if there is, in fact, anyone who still needs a spoiler warning for Spiderman one: The last ten minutes really spoilt it for me, skip 'em.

Comment author: CronoDAS 28 July 2009 07:57:23PM *  2 points [-]
Comment author: NQbass7 29 July 2009 03:03:09PM 0 points [-]

Well I WAS planning on getting some work done today.. but now...

Comment author: abigailgem 28 July 2009 09:04:19AM *  5 points [-]

Edit: major plot spoiler in this comment.

You miss out a major point of the story, that those who agree to sacrifice others' children are dishonourable, and that this matters; and that the main character, who sacrifices only one child to save all the rest (his grandchild) suffers terribly for this.

I would not argue from fictional evidence, but the storytellers seem keen to point this out. Also, when deciding to sacrifice children, all possible other courses of action must be eliminated first.

Edit: for me, the main interest of the trolley problem is the emotional response. Would you kill one to save five, if saving the five was certain if you killed the one, and impossible otherwise? Er, yes, I hope so, though I think such a situation, with such certainty, is unlikely. How do you feel about trolley problems generally? Horror and disgust. Then I see that even if I am not going to be in that situation, I may be in situations where I must behave rationally, and Stoically fight down emotional responses.

Comment author: nerzhin 28 July 2009 05:54:19PM 2 points [-]

How do you feel about trolley problems generally? Horror and disgust. Then I see that even if I am not going to be in that situation, I may be in situations where I must behave rationally, and Stoically fight down emotional responses.

I think of trolley problems as being related to Newcomb's problem.

If you expect to encounter Newcomb-like problems, you change your source code to precommit yourself to one-box, even though it seems less rational. Evolution expects us to encounter trolley-like problems, and has changed our source code (that's the "horror and disgust") so that we are in some sense precommitted to not throw the switch. And in general, if you're not considering theoretical dilemmas in an armchair, there are very good reasons for that source code change.

Comment author: taw 28 July 2009 03:48:09PM 4 points [-]

There have been many similar situations historically - food supply was more or less equal to food demand, so if food supply got suddenly lower for whatever reason, there wasn't enough food for everyone, and some people had to die.

The usual algorithm was that the poor people would be priced out of the food market, until enough of them died to restore Malthusian equilibrium. Most of the dead would be children.

How is that morally different from the situation described in the post?

Comment author: mikem 29 July 2009 03:20:38AM 2 points [-]

I can think of a couple of differences:

The poor people during a famine at least have a fighting chance, if slim. Somehow, by hook or by crook, attain money or food, or leave for a region where there is no famine.

Also, a famine is a matter of public knowledge, which allows the possibility for a society to collectively (or fragmentedly) come up with a solution. In the torchwood scenario, [small spoiler warning] the true nature of the threat and the solution devised by the executive branch were being kept a secret. In fact, they were actively suppressing groups who were moving for alternative stances towards the alien threat. If it were public knowledge, the to-be-sacrificed class would at least have the option of revolting against the powers/system/'algorithm' which was mandating their extermination.

Comment author: [deleted] 29 July 2009 07:30:50PM 1 point [-]

In the not-enough-food scenario, you have more bystander effect: the merchant doesn't necessarily feel like e's killing people by raising prices.

Comment author: mikem 28 July 2009 09:10:47AM 4 points [-]

Lesswrongers will be encouraged to learn that the Torchwood characters were rationalists to a man and woman - there was little hesitation in agreeing to the 456's demands.

Are you joking? They weren't rationalists, they were selfish. There is a distinction. They were looking after there own asses and those of their families (note that the complicit politicians specifically excluded their own family's children from selection, regardless of 'worth').

children - or units as they were plausibly referred to

What do you mean by 'plausibly'? They were referred to as units in order to dehumanize them. Because the people referring to the children as such recognized that what they were doing was abhorrently wrong, and so had to mask the fact, even to themselves, by obscuring the reality of what they were discussing: the wholesale slaughter of their fellows.

... governments paying attention to round up the orphans, the refugees and the unloved - for the unexpectedly rational reason of minimising the suffering of the survivors

That's laughable. It had nothing to do with minimizing suffering, that was a rationalization. They were doing it for the same reason any government targets the vulnerable; because there are few willing to protect them and argue for them. It was pretty clear if you watched the show that the children being targeted were hardly 'unloved'.

You can't consider the scenario without considering the precedent that it would set. The notion that there are wide swaths of the population -- children, who've never even had the opportunity to truly prove themselves or do much of anything -- who are completely without worth and sacrificeable at the whim of the government is untenable in a society that values things like individuality, personal autonomy, the pursuit of happiness and, well, human life! They would not be saving humanity, they would be mutilating it.

The poster failed to mention that the sacrificed children were being sentence to an eternal fate which worse than death.

And there is a difference between the actions of the government and the actions of the main character. One of them was fighting the monsters. The others were the monster's business partners.

Comment author: cousin_it 28 July 2009 09:40:28AM *  8 points [-]

The poster failed to mention that the sacrificed children were being sentenced to an eternal fate which worse than death.

I really wonder what other LWers will say about this. Would you prefer to give one person huge disutility, or destroy humankind? For extra fun consider a 1/2^^^3 chance of 3^^^3 disutility to that one person.

Eliezer in particular considers his utility function to be provably unbounded in the positive direction at least, thinks we have much more potential for pain than pleasure, thinks destroying humankind has finite disutility on the order of magnitude of "billions of lives lost" (otherwise he'd oppose the LHC no matter what), and he's an altruist and expected utility consequentialist. Together this seems to imply that he will have to get pretty inventive to avoid destroying humankind.

Comment author: Eliezer_Yudkowsky 10 September 2009 10:12:08PM 1 point [-]

1/2^^^3 = 2^^(2^^2) = 2^^(2^2) = 2^^4 = 2^2^2^2 = 65536.

Comment author: cousin_it 10 September 2009 11:03:38PM *  1 point [-]

Oh, shit. Well... uhhhh.. in the least convenient impossible possible world it isn't! :-)

Comment author: Larks 10 September 2009 10:22:02PM 1 point [-]

1/65536, surely?

Comment author: Eliezer_Yudkowsky 14 September 2009 06:25:12PM 0 points [-]

Er, yes.

Comment author: cousin_it 14 September 2009 04:53:37AM 0 points [-]

Come to think, I don't even see how your observation makes the question any easier.

?

Comment author: Eliezer_Yudkowsky 14 September 2009 06:26:22PM *  0 points [-]

1/65536 probability of someone suffering 3^^^3 disutilons? If humanity's lifespan is finite, that's far worse than wiping out humanity. (If humanity's lifespan is infinite, or could be infinite with probability greater than 1/65536, the reverse is true.)

Comment author: cousin_it 14 September 2009 06:39:36PM *  0 points [-]

I'll take that for an answer. Now let's go over the question again: if humanity's lifespan is potentially huge... counting "expected deaths from the LHC" is the wrong way to calculate disutility... the right way is to take the huge future into account... then everyone should oppose the LHC no matter what? Why aren't you doing it then - I recall you hoped to live for infinity years?

Comment author: CarlShulman 14 September 2009 06:56:38PM *  2 points [-]

The very small probability of a disaster caused directly by the LHC is swamped by the possible effects (positive or negative) of increased knowledge of physics. Intervening too stridently would be very costly in terms of existential risk: prominent physicists would be annoyed at the interference (asking why those efforts were not being dedicated to nuclear disarmament or biodefence efforts, etc) and could discredit concern with exotic existential risks (e.g. AI) in their retaliation.

Comment author: Eliezer_Yudkowsky 14 September 2009 07:17:47PM 2 points [-]

Agree with all except the first sentence.

Comment author: cousin_it 14 September 2009 10:27:10PM *  2 points [-]

...Okay. You do sound like an expected utility consequentialist, I didn't quite believe that before. Here's an upvote. One more question and we're done.

Your loved one is going to be copied a large number of times. Would you prefer all copies to get a dust speck in the eye, or one copy to be tortured for 50 years?

Comment author: CarlShulman 14 September 2009 07:23:03PM 1 point [-]

Hmm? In light of Bostrom and Tegmark's Nature article?

Comment author: Eliezer_Yudkowsky 14 September 2009 07:50:12PM 1 point [-]

We don't know enough physics to last until the end of time, but we know enough to build computers; if I made policy for Earth, I would put off high-energy physics experiments until after the Singularity. It's a question of timing. But I don't make such policy, of course, and I agree with the rest of the logic for why I shouldn't bother trying.

Comment author: rhollerith_dot_com 14 September 2009 08:49:03PM 0 points [-]

Same here. Unless I am missing something (and please do tell me if I am), the knowledge gained by the LHC is very unlikely to help much to increase the rationality of civilization or to reduce existential risk, so the experiment can wait a few decades, centuries or millenia till civilization has become vastly more rational (and consequently vastly better able to assess the existential risk of doing the experiment).

Comment author: botogol 29 July 2009 09:02:57AM 1 point [-]

What a lot of comments (and I was worried that it was all too trivial. Lesson: never underestimate the power of Dr Who) Thanks all.

@Nanani - yes, indeed, the initial round up of 600 or so was composed of waifs and strays like that, inc the ill. But when the demand of 10% was acceded to there wasn't time to handpick

@SharedPhoenix - I agree and a strength of this story was that was no easy way out. The scenario was played out right to the end with the main character forced to make a rational sacrifice. OK, he found a way for it to be jsut one child, but there was still a choice.

@mikem - I disagree. Yes there were selfish cabinet members simply looking out for their own (this was dealt with in several contexts - there was an assumption that the interests of one's own child is beyond the limit of human rationality) however the decision to accede to this, and actually make this a policy was taken by the prime minister for rational reasons. He recognised that unless he spared the children of the decision makers and enforcers, there would be no decisions and no enforcing. It was purely rational. (And 'units' yes I meant that it was a plausible that such a sinister euphemism would be employed)

@jwdink - yes, I was surprised they took that route (the rational give-in rather than fight to death) in TV-Land it was an unusual decision. That's why I wrote the post about it :-)

Comment author: thomblake 29 July 2009 06:47:18PM 1 point [-]

Threaded comments are your friend. Just click "reply" on the comment you want to reply to. Bonus: the relevant person will be notified of your response.

Comment author: jwdink 29 July 2009 03:00:32AM 1 point [-]

That's horrible. They should've fought the space monsters in an all out war. Better to die like that than to give up your dignity. I'm surprised they took that route on the show.

Comment author: SilasBarta 29 July 2009 04:20:01PM 1 point [-]

That's not as bad an idea as your post's -3 rating suggests. First of all, what's to ensure the aliens even keep their word? (I haven't seen this episode, so I don't know how that's handled.) For all we know, this could just be their way of "trolling" us so we get into an intraspecies flamewar and thus be unprepared for their actual plans, which are to attack and take whatever living children they can.

In that case, the "nuclear option" is to "kill the children before the aliens get to them" .. which ends the human race anyway. And if the human race is going to end anyway, why not take as many of them down with us as we can?

Comment author: jwdink 29 July 2009 04:37:54PM 0 points [-]

I'm surprised that was so downvoted too.

Perhaps I should rephrase it: I don't want to assert that it would've been objectively better for them to not give up the children. But can someone explain to me why it's MORE rational to give up in this situation?

Comment author: Aurini 02 August 2009 07:31:32AM 0 points [-]

I think it's my fault. I posted a... rather unpopular article about compromise.

I agree with your hawk/brinksmanship analysis of the strategy. I've found in life that 'the easy way out' is usually not so easy. I'm still trying to break it down into game-theory language appropriate for this site, however.

Comment author: Cyan 02 August 2009 04:11:46PM *  0 points [-]

By calling the downvotes your fault, it seems that you're asserting that your post is a causal ancestor of the downvotes, which is a bit off, I think. I'd guess that the downvotes of your post and the downvotes of jwdink's comment have a common cause (that being a certain utility-maximizing mindset and/or set of preferences), but are not otherwise causally related.

Comment author: thomblake 29 July 2009 06:14:15PM 0 points [-]

I agree with SilasBarta below. Resistance may be futile but we'll give them a hell of a fight. They won't get our children even over our dead bodies.

Comment author: NQbass7 29 July 2009 03:07:57PM 0 points [-]

How many lives is your dignity worth? Would you be willing to actually kill people for your dignity, or are you only willing to make that transaction if someone else is holding the knife?

Comment author: jwdink 29 July 2009 04:38:51PM *  2 points [-]

I don't quite understand how your rhetorical question is analogous here. Can you flesh it out a bit?

I don't think the notion of dignity is completely meaningless. After all, we don't just want the maximum number of people to be happy, we also want people to get what they deserve-- in other words, we want people to deserve their happiness. If only 10% of the world were decent people, and everyone else were immoral, which scenario would seem the more morally agreeable: the scenario in which the 10% good people were ensured perennial happiness at the expense of the other 90%'s misery, or the reversed scenario?

I'm just seeing something parallel here: it's not brute number of people living that matters, so much as those people having worthwhile existences. After sacrificing their children on a gamble, do these people really deserve the peace they get?

(Would you also assert that Ozymandias' decision in The Watchmen was morally good?)

Comment author: eirenicon 29 July 2009 06:07:12PM 1 point [-]

What do the space monsters deserve? If you factor in their happiness, it's an even more complicated problem. The space monsters need n human children to be happy. If you give them up, you have happy space monsters and (6 billion - n) happy (if not immediately, in the long term) humans. If you refuse, assuming the space monsters are unbeatable, you have happy space monsters and zero happy humans. The first scenario is better for both space monsters and humans. Sure, in the second scenario, the humans theoretically don't lose their dignity, but what does dignity mean to the dead?

To put it in another light, what if this situation happened a hundred years ago? Would you be upset that the people alive at the time caved in to the aliens' demands, or would you prefer the human race had been wiped out?

Comment author: jwdink 29 July 2009 07:13:08PM 0 points [-]

What do the space monsters deserve?

Haha, I was not factoring that in. I assumed they were evil. Perhaps that was close minded of me, though.

The first scenario is better for both space monsters and humans. Sure, in the second scenario, the humans theoretically don't lose their dignity, but what does dignity mean to the dead?

Some people would say that dying honorably is better than living dishonorably. I'm not endorsing this view, I'm just trying to figure out why it's irrational, while the utilitarian sacrifice of children is more rational.

To put it in another light, what if this situation happened a hundred years ago? Would you be upset that the people alive at the time caved in to the aliens' demands, or would you prefer the human race had been wiped out?

There are plenty of variables you can slide up and down to make one feel more or less comfortable with the scenario. But we already knew that, didn't we? That's what the original trolley problem tells us: that pushing someone off a bridge feels morally different than switching the tracks of a trolley. My concern is that I can't figure out how to call one impulse (the discomfort at destroying autonomy) an objectively irrelevant mere impulse, and another impulse (the comfort at preserving life) an objectively good fact. It seems difficult to throw just the bathwater out here, but I'd really like to preserve the baby. (See my other post above, in response to Nesov.)

Comment author: eirenicon 29 July 2009 08:02:58PM *  0 points [-]

Some people would say that dying honorably is better than living dishonorably. I'm not endorsing this view, I'm just trying to figure out why it's irrational, while the utilitarian sacrifice of children is more rational.

If a decision decreases [personal] utility, is it not irrational?

Some people would say that it is dishonourable to hand over your wallet to a crackhead with a knife. When I was actually in that situation, though (hint: not as the crackhead), I didn't think about my dignity. I just thought that refusing would be the dumbest, least rational possible decision. The only time I've ever been in a fight is when I couldn't run away. If behaving honourably is rational then being rational is a good way to get killed. I'm not saying that being rational always leads to morally satisfactory decisions. I am saying that sometimes you have to choose moral satisfaction over rationality... or the reverse.

As for the trolley problem, what we are dealing with is the aftermath of the trolley problem. If you save the people on the trolley, it could be argued that you have behaved dishonourably, but what about the people you saved? Surely they are innocent of your decision. If humanity is honourably wiped out by the space monsters, is that better than having some humans behave dishonourably and others (i.e. those who favoured resistance, but were powerless to effect it) survive honourably?

Comment author: jwdink 29 July 2009 08:12:57PM *  0 points [-]

If a decision decreases utility, is it not irrational?

I don't see how you could go about proving this.

As for the trolley problem, what we are dealing with is the aftermath of the trolley problem. If you save the people on the trolley, it could be argued that you have behaved dishonourably, but what about the people you saved? Surely they are innocent of your decision. If humanity is honourably wiped out by the space monsters, is that better than having some humans behave dishonourably and others (i.e. those who favoured resistance, but were powerless to effect it) survive honourably?

Well, wait. Are we dealing with the happiness that results in the aftermath, or are we dealing with the moral value of the actions themselves? Surely these two are discrete. Don't the intentions behind an action factor into the morality of the action? Or are the results all that matter? If intentions are irrelevant, does that mean that inanimate objects (entities without intentions, good or bad) can do morally good things? If a tornado diverts from a city at the last minute, was that a morally good action?

I think intentions matter. It might be the case that, 100 years later, the next generation will be happier. That doesn't mean that the decision to sacrifice those children was the morally good decision-- in the same way that, despite the tornado-free city being a happier city, it doesn't mean the tornado's diversion was a morally good thing.

Comment author: eirenicon 29 July 2009 08:44:26PM 1 point [-]

I should have said "decreases personal utility." When I say rationality, I mean rationality. Decreasing personal utility is the opposite of "winning".

Comment author: jwdink 29 July 2009 09:19:01PM 0 points [-]

Instrumental rationality: achieving your values.  Not necessarily "your values" in the sense of being selfish values or unshared values: "your values" means anything you care about.  The art of choosing actions that steer the future toward outcomes ranked higher in your preferences.  On LW we sometimes refer to this as "winning".

Couldn't these people care about not sacrificing autonomy, and therefore this would be a value that they're successfully fulfilling?

Comment author: Vladimir_Nesov 29 July 2009 09:27:48PM 0 points [-]

Yes they could care about either outcome. The question is whether they did, whether their true hidden preferences said that a given outcome is preferable.

Comment author: Vladimir_Nesov 29 July 2009 07:39:56PM 0 points [-]

Some people would say that dying honorably is better than living dishonorably. I'm not endorsing this view, I'm just trying to figure out why it's irrational, while the utilitarian sacrifice of children is more rational.

Utilitarian calculation is a more rational process of arriving at a decision, while for the output of this process (a decision) for a specific question you can argue that it's inferior to the output of some other process, such as free-running deliberation or random guessing. When you are comparing the decisions of sacrifice of children and war to the death, first isn't "intrinsically utilitarian", and the second isn't "intrinsically emotional".

Which of the decision is (actually) the better one depends on the preferences of one who decides, and preferences are not necessarily reflected well in actions and choices. It's instrumentally irrational for the agent to choose poorly according to its preferences. Systematic processes for decision-making allow agents to explicitly encode their preferences, and thus avoid some of the mistakes made with ad-hoc decision-making. Such systematic processes may be constructed in preference-independent fashion, and then given preferences as parameters.

Utilitarian calculation is a systematic process for computing a decision in situations that are expected to break intuitive decision-making. The output of a utilitarian calculation is expected to be better than an intuitive decision, but there are situations when utilitarian calculation goes wrong. For example, the extent to which you value things could be specified incorrectly, or a transformation that computes how much you value N things based on how much you value one thing may be wrong. In other cases, the problem could be reduced to a calculation incorrectly, losing important context.

However, whatever the right decision is, there normally should be a way to fix the parameters of utilitarian calculation so that it outputs the right decision. For example, if the right decision in the topic problem is actually war to the death, there should be a way to more formally understand the situation so that the utilitarian calculation outputs "war to the death" as the right decision.

Comment author: [deleted] 31 July 2009 03:35:05AM 0 points [-]

I'm not convinced utilitarian reasoning can always be applied to situations where two preferences come into conflict: Calculating "secondary" uncertain factors which could influence the value of each decision ruins the possibility of exactness. Even in the trolley problem, in all its simplicity, each decision has repercussions whose values have some uncertainty. Thus a decision doesn't always have a strict value, but a probable value distribution! We make a trolley decision by 1) Considering only so many iterations in trying to get a value distribution, and 2) seeing if there is a satisfying lack overlap between the two. When the two distributions overlap too much (and you know that they are approximate, due to the intractability of getting a perfect distribution), it's really a wild guess to say one decision is best.

Utilitarian calculation helps the process, by providing means of deciding when each value probability distribution is sharply enough defined, and whether the overlap meets your internal maximum overlap criteria (presuming that's sharply defined!), but no amount of reasoning can solve every moral dilemma a person might face.

Comment author: jwdink 29 July 2009 08:18:19PM 0 points [-]

Which of the decision is (actually) the better one depends on the preferences of one who decides

So if said planet decided that its preference was to perish, rather than sacrifice children, would this be irrational?

However, whatever the right decision is, there normally should be a way to fix the parameters of utilitarian calculation so that it outputs the right decision. For example, if the right decision in the topic problem is actually war to the death, there should be a way to more formally understand the situation so that the utilitarian calculation outputs "war to the death" as the right decision.

I don't see why I should agree with this statement. I was understanding a utilitarian calculation as either a) the greatest happiness for the greatest number of people or b) the greatest preferences satisfied for the greatest number of people. If a), then it seems like it might predictably give you answers that are at odds with moral intuitions, and have no way of justifying itself against these intuitions. If b), then there's nothing irrational about deciding to go to war with the aliens.

Comment author: Vladimir_Nesov 29 July 2009 08:38:01PM *  0 points [-]

So if said planet decided that its preference was to perish, rather than sacrifice children, would this be irrational?

You can't decide your preference, preference is not what you actually do, it is what you should do, and it's encoded in your decision-making capabilities in a nontrivial way, so that you aren't necessarily capable of seeing what it is.

Compare preference to a solution to an equation: you can see the equation, you can take it apart on the constituent terms, but its solution is nowhere to be found explicitly. Yet this solution is (say) uniquely defined by the equation, and approximate methods for solving the equation (analogized to the actual decisions) tend to give their results in the general ballpark of the exact solution.

Comment author: jwdink 29 July 2009 09:13:34PM 0 points [-]

You can't decide your preference, preference is not what you actually do, it is what you should do, and it's encoded in your decision-making capabilities in a nontrivial way, so that you aren't necessarily capable of seeing what it is.

You've lost me.

Comment author: Vladimir_Nesov 29 July 2009 09:18:53PM *  0 points [-]

The analogy in the next paragraph was meant to clarify. Do you see the analogy?

A person in this analogy is an equations together with an algorithm for approximately solving that equation. Decisions that the person makes are the approximate solutions, while preference is the exact solution hidden in the equation that the person can't solve exactly. The decision algorithm tries to make decisions as close to the exact solution as it can. The exact solution is what the person should do, while the output of the approximate algorithm is what the person actually does.

Comment author: Vladimir_Nesov 29 July 2009 05:44:49PM 1 point [-]
Comment author: jwdink 29 July 2009 07:05:28PM 1 point [-]

Yeah, the sentiment expressed in that post is usually my instinct too.

But then again, that's the problem: it's an instinct. If my utilitarian impulse is just another impulse, then why does it automatically outweigh any other moral impulses I have, such as a value of human autonomy? If my utilitarian impulse is NOT just an impulse, but somehow is objectively more rational and outranks other moral impulses, then I have yet to see a proof of this.

Comment author: Psy-Kosh 29 July 2009 07:47:52PM 1 point [-]

"shut up and multiply" is, in principle, a way to weigh various considerations like the value of autonomy, etc etc etc...

It's not "here's shut up and multiply" vs "some other value here", but "plug in your values + actual current situation including possible courses of action and compute"

Some of us are then saying "it is our moral position that human lives are so incredibly valuable that a measure of dignity for a few doesn't outweigh the massively greater suffering/etc that would result from the implied battle that would ensue from the 'battle of honor' route"

Comment author: jwdink 29 July 2009 08:08:13PM *  0 points [-]

Ah, then I misunderstood. A better way of phrasing my challenge might be: it sounds like we might have different algorithms, so prove to me that your algorithm is more rational.

No one has answered this challenge.

Comment author: Psy-Kosh 29 July 2009 05:37:00PM 0 points [-]

If you take an action that you know will result in a greater amount of death/suffering, just for the sake of your own personal dignity, do you actually deserve any dignity from that?

ie, one can rephrase the situation as "are you so selfish as to put your own personal dignity above many many human lives?" (note, I have not watched the Torchwood episodes in question, merely going at this based on the description here.)

IF fighting them or otherwise resisting is known to be futile and IF there's sufficient reason to suspect that they will keep their word on the matter, then the question becomes "just about everyone gets killed" vs "most survive, but some number of kids get taken to suffer, well, whatever the experience of being used as a drug is. (eventual death within a human lifespan? do they remain conscious long past that? etc etc etc...)"

That doesn't make the second option "good", but if the choices available amount to those two options, then we need to choose one.

"Everyone gets killed, but at least we get some 'warm fuzzies of dignity'" would actually seem to potentially be a highly immoral decision.

Having said that..... Don't give up searching for alternatives or ways to fight the monsters-in-question that doesn't result in automatic defeat. What's said above applies to the pathological dilemma in the least convenient possible world where we assume there really are no plausible alternatives.

Comment author: jwdink 29 July 2009 07:24:17PM 3 points [-]

Well, sure, when you phrase it like that. But your language begs the question: it assumes that the desire for dignity/autonomy is just an impulse/fuzzy feeling, while the desire for preserving human life is an objective good that is the proper aim for all (see my other posts above). This sounds probable to be me, but it doesn't sound obvious/ rationally derived/ etc.

I could after all, phrase it in the reverse manner. IF I assume that dignity/autonomy is objectively good:

then the question becomes "everyone preserves their objectively good dignity" vs. "just about everyone loses their dignity for destroying human autonomy, but we get that warm fuzzy feeling of saving some people." In this situation, "Everyone loses their dignity, but at least they get to survive--in the way that any other undignified organism (an amoeba) survives" would actually seem to be a highly immoral decision.

I'm not endorsing either view, necessarily. I'm just trying to figure out how you can claim one of these views is more rational or logical than the other.

Comment author: Psy-Kosh 29 July 2009 07:49:27PM 1 point [-]

Well then... I'd say a morality that puts the dignity of a few people (the decision makers) as having more importance than, well, the lives and well being of the majority of the human race is not a very good morality.

ie, I am claiming "it seems to be that a consequence of my morality is that..."

Alternately "sure, maybe you value 'battle of honor' more than human lives, but then your values don't seem to count as something I'd call morality"

Comment author: jwdink 29 July 2009 08:13:51PM 0 points [-]

Well then... I'd say a morality that puts the dignity of a few people (the decision makers) as having more importance than, well, the lives and well being of the majority of the human race is not a very good morality.

Okay. Would you say this statement is based on reason?

Comment author: Nanani 29 July 2009 01:07:51AM 1 point [-]

What if the children sacrificed are the ones who would expected to quickly die anyway? Young cancer patients, for instance, or any chld born with a defect that would have led to a very early death without modern medical care.

Their suffering would be diminished since it wouldn't last as long, and as a dark "plus", their parents were already facing the prospect of the child dying young.

I didn't see the show so I don't know if there was a caveat saying they had to be healthy children. Incidentally did the aliens describe the fate worse than death at all? Is it torture, or more like Borg-assimilation?

Comment author: Vladimir_Nesov 29 July 2009 11:51:46AM 1 point [-]

This criterion can't gather the required 10%.

Comment author: mikem 29 July 2009 03:07:28AM 1 point [-]

Spoiler warning!

The children were being incorporated into the bodies of the aliens. In the words of the rather terse (and uber-creepy) alien ambassador:

They create chemicals. The chemicals ... are good. We feel ... good. The chemicals are good.

Basically, they were being used as prosthetic glands secreting a narcotic, which the aliens found pleasant. That's why they wanted them. The aliens were shooting up, children. Somehow the (original 1965) children were preserved in their adolescent state, presumably indefinitely, melded into the alien's monstrous bodies. The nature of their subjective experience was left undisclosed, but it is hard to imagine it was pleasant, nor that, being reduced to glands, that they had any sort of autonomy.

So yeah, being a cancer patient or otherwise unhealthy child did not appear to be any 'advantage', at least not from the perspective of the child, since presumably their suffering existence would only be extended forever. Hard to see how the parents would see this as a plus either.

Comment author: gwern 28 July 2009 08:42:15AM 1 point [-]

"Answer: in a UK-centric political twist they chose those attending the schools with the poorest exam results."

Wait, what? Why not just go with those with the lowest scores period? Why use an indirect test like that (which probably has as much to do with socioeconomic status anyway)?

Comment author: ShardPhoenix 28 July 2009 10:40:34AM *  4 points [-]

Presumably mainly because it's satirical, but it'd also be a heck of a lot simpler to round people up all together at school than to pick them out one by one.

Comment author: whpearson 28 July 2009 12:27:42PM 0 points [-]

They were on a tight time schedule so it was this reason they used.