Eugine_Nier comments on Rationality Quotes November 2013 - Less Wrong

5 Post author: malcolmocean 02 November 2013 08:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (388)

You are viewing a single comment's thread.

Comment author: Eugine_Nier 02 November 2013 06:12:33AM 10 points [-]

Utilitarianism is not in our nature. Show me a man who would hold a child’s face in the fire to end malaria, and I will show you man who would hold a child’s face in the fire and entirely forget he was originally planning to end malaria.

James A. Donald

Comment author: FiftyTwo 04 November 2013 10:12:55PM *  32 points [-]

Medicine is not in our nature. Show me a man who would cut someone open to remove cancer, and I will show you man who would cut someone open and entirely forget he was originally planning to remove a tumour

Exact same argument. Does it sound equally persuasive to you?

Comment author: Jiro 05 November 2013 05:02:37PM *  10 points [-]

I'd extend Eugene's reply and point out that both the original and modified version of the sentence are observations. As such, it doesn't matter that the two sentences are grammatically similar; it's entirely possible that one is observed and the other is not. History has plenty of examples of people who are willing to do harm for a good cause and end up just doing harm; history does not have plenty of examples of people who are willing to cut people open to remove cancer and end up just cutting people open.

Also, the phrasing "to end malaria" isn't analogous to "to remove cancer" because while the surgery only has a certain probability of working, the uncertainty in that probability is limited. We know the risks of surgery, we know how well surgery works to treat cancer, and so we can weigh those probabilities. When ending malaria (in this example), the claim that the experiment has so-and-so chance of ending malaria involves a lot more human judgment than the claim that surgery has so-and-so chance of removing cancer.

Comment author: Desrtopa 08 November 2013 02:09:37AM 3 points [-]

History has plenty of examples of people who are willing to do harm for a good cause and end up just doing harm.

Yes, but keep in mind the danger of availability bias; when people are willing to do harm for a good cause, and end up doing more good than harm, we're not so likely to hear about it. Knut Haukelid and his partners caused the death of eighteen civilians, and may thereby have saved several orders of magnitude more. How many people have heard of him? But failed acts of pragmatism become scandals.

Also, some people (such as Hitler and Stalin) are conventionally held up as examples of the evils of believing that ends justify means, but in fact disavowed utilitarianism just as strongly as their critics. To quote Yvain on the subject, "If we're going to play the "pretend historical figures were utilitarian" game, it's unfair to only apply it to the historical figures whose policies ended in disaster."

Comment author: Jiro 08 November 2013 04:36:42PM 6 points [-]

We already have a situation where we can cause harm to innocent people for the general good. It's called taxes.

Since I got modded down for that before, here's a hopefully less controversial example: the penal system. If you decide that your society is going to have a penal system, you know (since the system isn't perfect) that your system will inevitably punish innocent people. You can try to take measures to reduce that, but there's no way you can eliminate it. Nobody would say we shouldn't put a penal system into effect because it is wrong to harm innocent people for the greater good--even though harming innocent people for the greater good is exactly what it will do.

I don't think anyone really objects to hurting innocent people for the greater good. The kind of scenarios that most people object to have other characteristics than just that and it may be worth figuring out what those are and why.

Also, some people (such as Hitler and Stalin) are conventionally held up as examples of the evils of believing that ends justify means, but in fact disavowed utilitarianism just as strongly as their critics.

It seems to me that utilitarianism decides how to act based on what course of action benefits people the most; deciding who counts as people is not itself utilitarian or non-utilitarian.

And even ignoring that, Hitler and Stalin may be valuable as examples because they don't resemble strict utilitarianism, but they do resemble utilitarianism as done by fallible humans. Actual humans who claim that the ends justify the means also try to downplay exactly how bad the end is, and their methods of downplaying that do resemble ideas of Hitler and Stalin.

Comment author: Desrtopa 11 November 2013 02:15:37AM *  5 points [-]

And even ignoring that, Hitler and Stalin may be valuable as examples because they don't resemble strict utilitarianism, but they do resemble utilitarianism as done by fallible humans. Actual humans who claim that the ends justify the means also try to downplay exactly how bad the end is, and their methods of downplaying that do resemble ideas of Hitler and Stalin.

Can you provide examples of this? In my experience, while utilitarianism done by fallible humans may be less desirable than utilitarianism as performed by ideal rationalists, the worst failures of judgment on an "ends justify the means" basis tend not to come from people actually proposing policies on a utilitarian basis, but from people who were not utilitarians whose policies are later held up as examples of what utilitarians would do, or from people who are not utilitarians proposing hypotheticals of their own as what policies utilitarianism would lead to.

Non utilitarians in my experience generally point to dangers of a hypothetical "utilitarianism as implemented by someone much dumber or more discriminatory than I am," which is why for example in Yvain's Consequentialism FAQ, the objections he answered tended to be from people believing that utilitarians would engage in actions that those posing the objections could see would lead to bad consequences.

Utilitarianism as practiced by fallible humans would certainly have its failings, but there are also points of policy where it probably offers some very substantial benefits relative to our current norms, and it's disingenuous to focus only on the negative or pretend that humans are dumber than they actually are when it comes to making utilitarian judgments.

Comment author: Jiro 30 November 2013 03:28:28AM -1 points [-]

Any example I could give you of humans fallibly being utilitarian you could equally well describe as an example of humans not being utilitarian at all. After all, that's what "fallible" means--"doing X incorrectly" is a type of "not doing X".

If you want an example of humans doing something close to utilitarian (which is all you're going to get, given how the word "fallible" works), Stalin himself is an example. Just about everything he did was described as being for the greater good, because building the perfect Soviet society is for the greater good and the harm done to someone by giving him a show trial and executing him is necessary to build that society. Of course, you could explain how Stalin wasn't really utilitarian, but if he was really utilitarian, he wouldn't be fallibly utilitarian.

Comment author: Desrtopa 30 November 2013 07:47:40PM 0 points [-]

I would accept any figure as "fallibly utilitarian" if they endorsed utilitarian ethics and claimed to be attempting to follow it, but Stalin did not do so, and while his actions might be interpreted as a fallible attempt at utilitarianism, his own pronouncements don't particularly invite such interpretation.

Comment author: Jiro 30 November 2013 11:57:35PM -1 points [-]

That doesn't actually contain any of his own pronouncements.

I managed to Google this for Hitler: "It is thus necessary that the individual should come to realize that his own ego is of no importance in comparison with the existence of his nation; that the position of the individual ego is conditioned solely by the interests of the nation as a whole ... that above all the unity of a nation's spirit and will are worth far more than the freedom of the spirit and will of an individual. .... This state of mind, which subordinates the interests of the ego to the conservation of the community, is really the first premise for every truly human culture .... we understand only the individual's capacity to make sacrifices for the community, for his fellow man."

This may not be technically utilitarian,. but it is an example of Hitler endorsing the idea that some people should be harmed to benefit others (in this case to benefit society), and is an example of how that doesn't go well.

Comment author: Desrtopa 01 December 2013 01:33:52AM *  0 points [-]

People have been proposing that some people should be harmed to benefit others long before anyone proposed the idea of utilitarianism; usually it was justified because the people being harmed are outgroup members, or simply Less Important compared to the people being helped, or to subordinate individual identity to the group identity.

Sometimes, this doesn't go well. In some cases, such as, by your own example, in the case of taxes or a justic system, it goes much better than a refusal to harm some people to help others. Utilitarianism is a construction for formalizing under what conditions it is or is not a good idea to attempt such tradeoffs. Calling the purges of Hitler or Stalin failings of Utilitarianism is about as fair as calling every successful government intervention or institution ever, which after all all involve sacrificing resources of the public for a common good, successes of Utilitarianism.

Comment author: NancyLebovitz 12 November 2013 11:59:09AM 4 points [-]

Another way that a penal system is extremely likely to harm innocents is that the imprisoned person may have been supplying a net benefit to their associates in non-criminal ways, and they can't continue to supply those benefits while in prison. This is especially likely for some of the children of prisoners, even if the prisoners were guilty..

Comment author: Eugine_Nier 13 November 2013 04:09:37AM -2 points [-]

That's indirect harm. Non-utilitarians don't have to care about it (and certainly not care about it as much as utilitarians).

Comment author: TheOtherDave 08 November 2013 08:03:16PM 4 points [-]

Nobody would say we shouldn't put a penal system into effect because it is wrong to harm innocent people for the greater good--even though harming innocent people for the greater good is exactly what it will do.

I am unsure how to map decisions under uncertainty to evidence about values as you do here.

A still-less-controversial illustration: I am shown two envelopes, and I have very high confidence that there's a $100 bill in exactly one of those envelopes. I am offered the chance to pay $10 for one of those envelopes, chosen at random; I estimate the EV of that chance at $50, so I buy it. I am then (before "my" envelope is chosen) offered the chance to pay another $10 for the other envelope, this chance to be revoked once the first envelope is selected. For similar reasons I buy that too.

I am now extremely confident that I've spent $10 for an empty envelope... and I endorse that choice even under reflection. But it seems ridiculous to conclude from this that I endorse spending $10 for an empty envelope. Something like that is true, yes, but whatever it is needs to be stated much more precisely to avoid being actively deceptive.

It seems to me that if I punish a hundred people who have been convicted of a crime, even though I'm confident that at least some of those people are innocent, I'm in a somewhat analogous situation to paying $10 for an empty envelope... and concluding that I endorse punishing innocent people seems equally ridiculous. Something like that is true, yes, but whatever it is needs to be stated much more precisely to avoid being actively deceptive.

Comment author: Jiro 08 November 2013 09:33:37PM 4 points [-]

In your example, you are presenting "I think you should spend $10 for an empty envelope" as a separate activity, and you are being misleading because you are not putting it into context and saying "I think you should spend $10 for an empty envelope, if this means you can get a full one".

With the justice system example, I am presenting the example in context--that is, I am not just saying "I think you should harm innocent people", I am saying "I think you should harm innocent people, if other people are helped more". It's the in-context version of the statement that I am presenting, not the out-of-context version.

Comment author: TheOtherDave 08 November 2013 09:47:48PM 1 point [-]

(nods) Yes, that makes sense.
Thanks.

Comment author: Eugine_Nier 10 November 2013 08:19:47PM *  2 points [-]

I (and James Donald) agree. Remember that the traditional ethical laws this is based on also have traditional exceptions, e.g., for punishment and war, and additional laws governing when and how those exceptions apply. The thing to remember is that you are not allowed to add to the list of exceptions as you see fit, nor are you allowed to play semantic games to expand them. In particular, no "war on poverty", or "war on cancer", even "war on terror" is pushing it.

Comment author: Jiro 11 November 2013 04:32:37PM 1 point [-]

I think you're misunderstanding me. We all know that most ethical systems think it's okay to punish criminals. I'm not referring to the fact that criminals are punished, but the fact that when we try to punish criminals we will, since no system is perfect, inevitably end up punishing some innocent people as well. Those people did nothing wrong, yet we are hurting them, and for the greater good.

Comment author: Eugine_Nier 12 November 2013 01:40:35AM *  1 point [-]

This is no different from the fact that it's okay to fly planes even though some of them will inevitably crash.

Note that if a judge punishes someone who turns out to be innocent, we believe he should feel guilty about this rather then simply shrugging and saying "mistakes will happen". Similarly, if an engeneer makes a mistake than causes a plane to crash.

Comment author: Jiro 12 November 2013 03:20:21PM 1 point [-]

Just like not all people punished are guilty, not all innocent people punished are discovered; there's always going to be a certain residue of innocent people who are punished, but not discovered, with no guilty judges or anything else to make up for it. Hurting such innocent people is nevertheless an accepted part of having a penal system.

Comment author: Eugine_Nier 05 November 2013 01:41:01AM -2 points [-]

The second sentence is an empirical observation that is clearly false in your example.

Comment author: lmm 17 November 2013 11:20:00PM *  1 point [-]

Yep. I've heard similar speculations regarding surgeons before. Fortunately nowadays we can take appropriate measures to compensate (surgeons are highly paid and closely monitored; we take a lot of care that medicine be evidence-based; the rationale behind specific medical interventions is carefully documented and checked multiple times; we require medical professionals to train for longer than any other profession). But note that for most of human history, the interventions performed by almost all medical professionals were literally worse than nothing.

Comment author: wiresnips 03 November 2013 07:05:40PM 10 points [-]

Utilitarianism isn't a description of human moral processing, it's a proposal for how to improve it.

Comment author: Eugine_Nier 04 November 2013 04:12:22AM 14 points [-]

One problem is that if we, say, start admiring people for acting in "more utilitarian" ways, what we may actually be selecting for is psychopathy.

Comment author: wiresnips 04 November 2013 06:15:33AM 7 points [-]

Agreed. Squicky dilemmas designed to showcase utilitarianism are not generally found in real life (as far as I know). And a human probably couldn't be trusted to make a sound judgement call even if one were found. Running on untrusted hardware and such.

Ah- and this is the point of the quote. Oh, I like that.

Comment author: DanielLC 02 November 2013 11:20:05PM 2 points [-]

Our nature is not purely utilitarian, but I wouldn't go so far as to say that utilitarianism is not in our nature. There are things we avoid doing regardless of how they advance our goals, but most of what we do is to accomplish goals. If you can't understand that there are things you need to do to eat, then you won't eat.

Comment author: Eugine_Nier 03 November 2013 04:17:26AM *  0 points [-]

Strawman. Does any moral system anyone's ever proposed say we should never attempt to accomplish goals?

Comment author: Jayson_Virissimo 02 November 2013 05:45:26PM 2 points [-]

I agree that utilitarianism is "not in our nature," but what has this to do with rationality?

Comment author: RichardKennaway 02 November 2013 06:49:56PM 4 points [-]

I agree that utilitarianism is "not in our nature," but what has this to do with rationality?

Utilitarianism is pretty fundamental around here. Not everyone here agrees with it, but pretty much all ethical discussions here take it as a precondition for even having a discussion. The assertion that we are not, cannot be, and never will be utilitarians is therefore very relevant.

If you are suggesting by that emphasis on "nature" that we might act to change our nature and remake ourselves into better utilitarians, I would ask, if we are in fact not utilitarians, why should we make ourselves so? Infatuation with the tidiness of the VNM theorem?

Comment author: Mestroyer 02 November 2013 11:32:52PM 8 points [-]

We us::should try to be as utilitarian as we can because our intuitive morality is kind of consequentialist, so we care about how the world actually ends up, and utilitarianism helps us win.

If we ever pass up a chance to literally hold one child's face to a fire and end malaria, we have screwed up. We are not getting what we care about most.

It's not the "tidiness" in any aesthetic sense of VNM axioms that are important, it's the not-getting-money-pumped. Not being able to be money pumped is important not because getting money pumped is stupid and we can't be stupid, but because we need to use our money on useful stuff.

Comment author: Eugine_Nier 03 November 2013 04:21:53AM *  1 point [-]

If we ever pass up a chance to literally hold one child's face to a fire and end malaria, we have screwed up.

In another comment James A. Donald suggests a way torturing children could actually help cure malaria:

To cure malaria, we really need to experiment on people. For some experiments, obtaining volunteers is likely to be difficult, and if one experimented on non volunteering adults, they would probably create very severe difficulties. Female children old enough to have competent immune systems, but no older, would be ideal.

Would you be willing to endorse this proposal? If not, why not?

Comment author: Armok_GoB 04 November 2013 01:41:36AM *  9 points [-]

If I'm not fighting the hypothetical, yes I would.

If I encountered someone claiming that in the messy real world, then I run the numbers VERY careful and most likely conclude the probability is infinitesimal of him actually telling the truth and being sane. Specifically, of those claims the one that it'd be easier to kidnap someone than to find volunteer (say, adult willing to do it in exchange for giving their families large sums of money) sounds highly implausible.

Comment author: Vaniver 04 November 2013 02:32:30AM 3 points [-]

Specifically, of those claims the one that it'd be easier to kidnap someone than to find volunteer (say, adult willing to do it in exchange for giving their families large sums of money) sounds highly implausible.

What's your opinion of doing it Tuskegee-style, rather than kidnapping them or getting volunteers? (One could believe that there might be a systematic difference between people who volunteer and the general population, for example.)

Comment author: Armok_GoB 04 November 2013 06:16:35AM 2 points [-]

That already had a treatment, hence it was not going to save the millions suffering, since they were already saved. Also, those scientist didn't have good enough methodology to have gotten anything useful out of it in either case. There's a general air of incompetence surrounding the whole thing that worries me more than the morality.

As I said; before doing anything like this you have to run your numbers VERY carefully. The probability of any given study solving a disease on it's own is extremely small, and there are all sorts of other practical problems. That's the thing; utilitarianism is correct, and not answering according to it is fighting the hypothetical. but in cases like this perhaps you should fight the hypothetical, since you're using specific historical examples that very clearly did NOT have positive utility and did NOT run the numbers.

It's a fact that a specific type of utilitarianism is the only thing that makes sense if you know the math. It's also a fact that there are many if's and buts that make human non-utilitarian moral intuition an heuristic way more reliable for actually achieving the greatest utility than trying to run the numbers yourself in the vast majority of real world cases. Finally, it's a fact that most things done in the name of ANY moral system is actually bullshit excuses.

Comment author: NancyLebovitz 04 November 2013 01:27:11PM 5 points [-]

http://en.wikipedia.org/wiki/Tuskegee_syphilis_experiment

Several African American health workers and educators associated with Tuskegee Institute helped the PHS to carry out its experimentation and played a critical role in its progression, though the extent to which they were aware of methodology of the study is not clear in all cases. Robert Russa Moton, the head of Tuskegee Institute at the time, and Eugene Dibble, of the Tuskegee Medical Hospital, both lent their endorsement and institutional resources to the government study. Nurse Eunice Rivers, an African-American trained at Tuskegee Institute who worked at its affiliated John Andrew Hospital, was recruited at the start of the study.

Vonderlehr was a strong advocate for Nurse Rivers' participation, as she was the direct link to the community. During the Great Depression of the 1930s, the Tuskegee Study began by offering lower class African Americans, who often could not afford health care, the chance to join "Miss Rivers' Lodge". Patients were to receive free physical examinations at Tuskegee University, free rides to and from the clinic, hot meals on examination days, and free treatment for minor ailments.

Based on the available health care resources, Nurse Rivers believed that the benefits of the study to the men outweighed the risks.

What do you think of that utilitarian calculation? I'm not sure what I think of it.

Comment author: gjm 04 November 2013 03:47:16PM 3 points [-]

It seems like either (1) Rivers was deceived, or (2) she was in some other way unaware that there was already an effective cure for syphilis which was not going to be given to the experimental subjects, or (3) the other options available to these people were so wretched that they were worse than having syphilis left untreated.

In cases 1 and 2, it doesn't really matter what we think of her calculations; if you're fed sufficiently wrong information then correct algorithms can lead you to terrible decisions. In case 3, maybe Rivers really didn't have anything better to do -- but only because other circumstances left the victims of this thing in an extraordinarily terrible position to begin with. (In much the same way as sawing off your own healthy left arm can be the best thing to do -- if someone is pointing a gun at your head and will definitely kill you if you don't. That doesn't say much about the merits of self-amputation in less ridiculous situations.)

I find #3 very implausible, for what it's worth.

(Now, if the statement were that Rivers believed that the benefits to the community outweighed the risks, and indeed the overt harm, to the subjects of the experiment, that would be more directly to the point. But that's not what the article says.)

Comment author: Lumifer 04 November 2013 05:28:26PM 1 point [-]

What do you think of that utilitarian calculation?

Which one? The presumed altruistic one or the real-life one (which I think included the utilitly of having a job, the readiness to disobey authority, etc.)

Comment author: Desrtopa 05 November 2013 04:51:24PM *  1 point [-]

In general, given ethical norms as they currently exist, rather than in a hypothetical universe where everyone is a strict utilitarian, I think the expected returns on such an experiment are unlikely to be worth the reputational costs.

The Tuskegee experiment may have produced some useful data, but it certainly didn't produce returns on the scale of reducing global syphilis incidence to zero. Likewise, even extensive experimentation on abducted children is unlikely to do so for malaria. The Tuskegee experiment though, is still seen as a black mark on the reputation of medical researchers and the government; I've encountered people who, having heard of it, genuinely believed that it, rather than the extremely stringent standards that currently exist for publishable studies, was a more accurate description of the behavior of present researchers. That sort of thing isn't easy to escape.

Any effective utilitarian must account for the fact that we're operating in a world which is extremely unforgiving of behavior such as cutting up a healthy hospital visitor to save several in need of organ transplants, and condition their behavior on that knowledge.

Comment author: NancyLebovitz 06 November 2013 06:08:01PM 9 points [-]

Here's one with actual information gained: Imperial Japanese experimentation about frostbite

For example, Unit 731 proved that the best treatment for frostbite was not rubbing the Limb, which had been the traditional method but immersion in water a bit warmer than 100 degrees, but never mom than 122 degrees.

The cost of this scientific breakthrough was borne by those seized for medical experiments. They were taken outside and left with exposed arms, periodically drenched with water, until a guard decided that frostbite had set in. Testimony From a Japanese officer said this was determined after the "frozen arms, when struck with a short stick, emitted a sound resembling that which a board gives when it is struck."

I don't get the impression that those experiments destroyed a lot of trust-- nothing compared to the rape of Nanking or Japanese treatment of American prisoners of war.

However, it might be worth noting that that sort of experimentation doesn't seem to happen to people who are affiliated with the scientists or the government.

Logically, people could volunteer for such experiments and get the same respect that soldiers do, but I don't know of any real-world examples.

Comment author: Jiro 06 November 2013 06:48:27PM 2 points [-]

I don't get the impression that those experiments destroyed a lot of trust-- nothing compared to the rape of Nanking or Japanese treatment of American prisoners of war.

It's hard for experiments to destroy trust when those doing the experiments aren't trusted anyway because they do other things that are as bad (and often on a larger scale).

Comment author: Desrtopa 08 November 2013 02:19:53AM 1 point [-]

Logically, people could volunteer for such experiments and get the same respect that soldiers do, but I don't know of any real-world examples.

I was going to say that I didn't think that medical researchers had ever solicited volunteers for experiments which are near certain to produce such traumatic effects, but on second thought, I do recall that some of the early research on the effects of decompression (as experienced by divers) was done by a scientist who solicited volunteers to be subjected to decompression sickness. I believe that some research on the effects of dramatic deceleration was also done similarly.

Comment author: fubarobfusco 06 November 2013 06:50:05PM 2 points [-]

Actual medical conspiracies, such as the Tuskegee syphilis experiment, probably contribute to public credence in medical conspiracy theories, such as anti-vax or HIV-AIDS denialism, which have a directly detrimental effect on public health.

Comment author: Desrtopa 06 November 2013 07:05:59PM 3 points [-]

Probably.

In a culture of ideal rationalists, you might be better off having a government run lottery where people were randomly selected for participation in medical experiments, with participation on selection being mandatory for any experiment, whatever its effects on the participants, and all experiments being vetted only if their expected returns were more valuable than any negative effect (including loss of time) imposed on the participants. But we're a species which is instinctively more afraid of sharks than stairs, so for human beings this probably isn't a good recipe for social harmony.

Comment author: linkhyrule5 03 November 2013 05:02:28AM 2 points [-]

The question is not "would this be a net benefit" (and it probably would, as much as I cringe from it). The question is, are there no better options?

Comment author: Eugine_Nier 03 November 2013 05:11:18AM *  1 point [-]

The question is, are there no better options?

Such as? Experimenting on animals? That will probably cause progress to be slower and think about all the people who would die from malaria in the meantime.

Comment author: linkhyrule5 03 November 2013 07:43:00AM *  3 points [-]

Yes. How many more? Would experimenting on little girls actually help that much? Also consider that many people consider a child's life more valuable than an adult one, that even in a world where you would not have to kidnap girls and evade legal problems and deal with psychological costs on the scientists caring for little humans is significantly more expensive then caring for little mice, that said kidnapping, legal, and psychological costs do exist, that you could instead spend that money on mosquito nets and the like and save lives that way...

The answer is not obviously biased towards "experiment on little girls.". In fact, I'd say it's still biased towards "experiment on mice." Morality isn't like physics, the answer doesn't always add up to normality, but a whole lot of the time it does.

Comment author: RichardKennaway 03 November 2013 08:30:39AM *  12 points [-]

Would experimenting on little girls actually help that much?

...

The answer is not obviously biased towards "experiment on little girls.". In fact, I'd say it's still biased towards "experiment on mice."

So your answer is that in fact it would not work. That is a reasonable response to an outrageous hypothetical. Yet James A. Donald suggested a realistic scenario, and beside it, the arguments you come up with look rather weak.

Would experimenting on little girls actually help that much? Also consider that many people consider a child's life more valuable than an adult one

Given the millions killed by malaria and at most thousands of experimental subjects, it takes a heavy thumb on the scales of this argument to make the utilitarian calculation come out against.

...evade legal problems and deal with psychological costs...

This is a get-out-of-utilitarianism-free card. A real utilitarian simply chooses the action of maximum utility. He would only pay a psychological cost for not doing that. When all are utilitarians the laws will also be utilitarian, and an evaluation of utility will be the sole criterion applied by the courts.

You are not a utilitarian. Neither is anyone else. This is why there would be psychological costs and why there are legal obstacles. You feel obliged to pretend to be a utilitarian, so you justify your non-utilitarian repugnance by putting it into the utilitarian scales.

caring for little humans is significantly more expensive then caring for little mice

But not any more expensive than caring for chimpanzees. Where, of course, "care for" does not mean "care for", but means "keep sufficiently alive for experimental purposes".

This looks like motivated reasoning. The motivation, to not torture little children, is admirable. But it is misapplied.

Morality isn't like physics

Can you expand on what you see as the differences?

Comment author: Armok_GoB 04 November 2013 01:51:18AM 0 points [-]

Of wait we're talking about an entire society thats utilitarian and rational. In that case I'm (coordinating with everyone else via auman agreement) just dedicating the entire global population to a monstrous machine for maximally efficient FAI research where 99% of people are suffering beyond comprehension with no regard for their own well being in order to support a few elite researchers as the dedicate literally every second of their lives to thinking at maximal efficiency while pumped full of nootropics that'll kill them in a few years.

Comment author: Desrtopa 08 November 2013 02:46:18AM 1 point [-]

Given the millions killed by malaria and at most thousands of experimental subjects, it takes a heavy thumb on the scales of this argument to make the utilitarian calculation come out against.

If it would result in a timely cure for malaria which would result in the disease's global eradication or near-eradication, I would say that it would be worth kidnapping a few thousand children. But not only would a world where you could get away with doing so differ from our own in some very significant ways, I honestly doubt that a few thousand captive test subjects constitute a decisive and currently limiting factor in the progress of the research.

Comment author: linkhyrule5 03 November 2013 10:37:31PM 1 point [-]

Would experimenting on little girls actually help that much?

...

No, seriously. I've read the original comment, James A. Donald does not support his claim.

But not any more expensive than caring for chimpanzees. Where, of course, "care for" does not mean "care for", but means "keep sufficiently alive for experimental purposes".

This is granted. References to small mice were silly and are now being replaced by "small chimpanzees." However...

Given the millions killed by malaria and at most thousands of experimental subjects, it takes a heavy thumb on the scales of this argument to make the utilitarian calculation come out against.

This is not the calculation being made. Using your numbers, experimenting on little girls needs to be at least 1.001 times as effective as experimenting on chimpanzees or mice to be worthwhile (because then you save an extra thousand lives for your thousand girls sacrificed.) It's not a flat "little girls versus millions of malaria deaths."

This is, quite frankly, not clear to me, and I'd want to call in an actual medical researcher to clarify. Doubly so, with artificial human organs becoming more and more possible (such organs are obviously significantly cheaper than humans.)

This is a get-out-of-utilitarianism-free card. A real utilitarian simply chooses the action of maximum utility. He would only pay a psychological cost for not doing that. When all are utilitarians the laws will also be utilitarian, and an evaluation of utility will be the sole criterion applied by the courts.

Actually, I was interpreting the hypothetical as "utilitarian government in our world." But fine, least convenient possible world and all that. That's why I set the non-society costs aside from the rest.

You feel obliged to pretend to be a utilitarian, so you justify your non-utilitarian repugnance by putting it into the utilitarian scales.

This looks like motivated reasoning. The motivation, to not torture little children, is admirable. But it is misapplied.

Honestly, this is probably true - case in point, I would rather not write a similar post from the opposite side. That being said, looking through my arguments, most of them hinge on the implausibility of human experimentation really being all that more effective compared to chimpanzee and artificial organ experimentation.

Morality isn't like physics

Can you expand on what you see as the differences?

The physics calculations around us have already been done perfectly. If, when we try to emulate them with our theories, we get something abnormal, it means our calculations are wrong and we need to either fix the calculation or the model. When we've done it all right, it should all add up to normality.

Our current morality, on the other hand, is a thing created over a few thousand years by society as a whole, that occasionally generates things like slavery. It is not guaranteed to already be perfectly calculated, and if our calculations turn out something abnormal, it could mean that either our calculations or the world is wrong.

Comment author: Mestroyer 03 November 2013 04:40:21AM 3 points [-]

Endorse? You mean, publicly, not on LessWrong, where doing so will get me much more than downvotes, and still have zero chance of making it actually happen? Of course not, but that has nothing to do with whether it's a good idea.

Comment author: Eugine_Nier 03 November 2013 05:05:52AM *  0 points [-]

I meant "endorse" in the sense that, unlike the Milgram experiment, there is no authority figure to take responsibility on your behalf.

Do you think it's a good idea?

Comment author: Mestroyer 03 November 2013 05:15:07AM 5 points [-]

If it will actually work, and there's no significant (as in at least the size of malaria being cured faster), and bad, consequences we're missing, or there are significant bad consequences but they're balanced out by significant good consequences we're missing, then yes.

Comment author: Armok_GoB 04 November 2013 01:36:32AM 0 points [-]

Then risk being the later man, while taking as many precautions as possible to preserve your intent.