Emile comments on Human errors, human values - Less Wrong

32 Post author: PhilGoetz 09 April 2011 02:50AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (135)

You are viewing a single comment's thread. Show more comments above.

Comment author: Emile 12 April 2011 06:42:55AM 6 points [-]

There's probably a least convenient possible world in which I'd bite the bullet and agree that it might be right for the doctor to kill the patient.

Suppose that on planets J and K, doctors are robots, and that it's common knowledge that they are "friendly" consequentialists who take the actions that maximize the expected health of their patients ("friendly" in the sense that they are "good genies" whose utility function matches human morality, i.e. they don't save the life of a patient that wants to die, don't value "vegetables" as much, etc.).

But on planet J, robot doctors treat each patient in isolation, maximizing his expected health, whereas on planet K doctors maximize the expected health of their patients as a whole, even if that means killing one to save five others.

I would prefer to live on planet K than on planet J, because even if there's a small probability p that I'll have my organs harvested to save five other patients, there's also a probability 5 * p that my life will be saved by a robot doctor's cold utilitarian calculation.

Comment author: DSimon 12 April 2011 02:15:54PM *  0 points [-]

"friendly" in the sense that they are "good genies" whose utility function matches human morality, i.e. they don't save the life of a patient that wants to die, don't value "vegetables" as much, etc.

Does this include putting less value on patients who would only live a short while longer (say, a year) with a transplant than without? AIUI this is typical of transplant patients.

Comment author: Emile 12 April 2011 02:58:04PM 2 points [-]

Probably yes, which would mean that in many cases the sacrifice wouldn't be made (though - least convenient possible world again - there are cases where it would).