[SEQ RERUN] Torture vs. Dust Specks

MinibearRex

Today's post, Torture vs. Dust Specks was originally published on 30 October 2007. A summary (taken from the LW wiki):

If you had to choose between torturing one person horribly for 50 years, or putting a single dust speck into the eyes of 3^^^3 people, what would you do?

Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Motivated Stopping and Motivated Continuation, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.

Today's post, Torture vs. Dust Specks was originally published on 30 October 2007. A summary (taken from the LW wiki):

If you had to choose between torturing one person horribly for 50 years, or putting a single dust speck into the eyes of 3^^^3 people, what would you do?

You know what? You are absolutely right.

If the background rate at which dust specks enter eyes is, say, once per day, then an additional dust speck is barely even noticeable. The 3^^^3 people probably wouldn't even be able to tell that they got an "extra" dust speck, even if they were keeping an excel spreadsheet and making entries every time they got a dust speck in their eye, and running relevant statistics on it. I think I just switched back to SPECKS. If a person can't be sure that something even happened to them, my utility function is rounding it off to zero.

If a person can't be sure that something even happened to them, my utility function is rounding it off to zero.

This may be already obvious to you, but such a utility function is incoherent (as made vivid by examples like the self-torturer).

1occlude15y

I expect that more than one of my brain modules are trying to judge between incompatible conclusions, and selectively giving attention to the inputs of the problem. My thinking was similar to yours -- it feels less like I'm applying scope insensitivity and more like I'm rounding the disutility of specks down due to their ubiquity, or their severity relative to torture, or the fact that the effects are so dispersed. If one situation goes unnoticed, lost in the background noise, while another irreparably damages someone's mind, then that should have some impact on the utility function. My intuition tells me that this justifies rounding the impact of a speck down to zero, that the difference is a difference of kind, not of degree, that I should treat these as fundamentally different. At the same time, like Vincent, I'm inclined to assign non-zero disutility value to a speck. One brain, two modules, two incompatible judgements. I'm willing to entertain the possibility that this is a bug. But I'm not ready yet to declare one module the victor.

9

[SEQ RERUN] Torture vs. Dust Specks

9

9

9

[SEQ RERUN] Torture vs. Dust Specks

9

9