I wonder if there is any right answer. We are talking about moral intuitions, feelings, which come from some presumably algorithmic part of our brain which seems it can be influenced, at least somewhat changed, by our rational minds. When we ask the question: which is worse 3^^^3 dust specks of 50 years of torture, are we asking what our feeling would be when confronted with a real world in which somehow we could see this happening? As if we could ever see 3^^^3 of anything? Because that seems to be the closest I can get to defining right and wrong.
Yes, the brain is a great tool for running hypotheticals to predict how the real thing might go. So we are running hypotheticals of 3*3 dust specks and hypotheticals of 50 years of torture, and we are trying to see how we feel about them?
I think it is not that hard to "break" the built-in for determining how one feels morally about various events. The thing was designed for real world circumstances and honed by evolution to produce behavior that would cause us to cooperate in a way that would ensure our survival, enhance our productivity and reproductivity. When we get to corner cases that clearly the original moral systems could not have been designed for, because NOTHING like them ever happened in our evolution, what can we say about our moral intuition? Even if it would make sense to say there is one right answer as to how our moral intuition "should" work in this corner case, why would we put any stock behind it?
I think in real life, the economic forces of a gigantic civilization of humans would happily torture a small number of individuals for 50 years if it resulted in the avoidance of 3^^^3 dust specks. We build bridges and buildings where part of the cost is a pretty predictable loss of human life, and a predictable amount of that being painful and a predictable amount of crippling, from construction accidents. We drive and fly and expose ourselves to carcinogens. We expose others to carcinogens.
In some important sense, when I drive a car that costs $10,000 instead of one that costs $8000, but I send the extra $2000 to Africa to relieve famine, I am trading some number of African starvations for leather seats and a nice stereo.
What is the intuition I am supposed to be pumping with dust specks? I think it might be that how you feel morally about things that are way beyond the edges of the environment in which your moral intuitions evolved are not meaningfully either moral or immoral.
Givewell does not think you can save one African from starving with $2000. You might be able to save one child from dying of malaria via insecticide-treated mosquito bednets. But this of course will not be the optimal use of $2K even on conventional targets of altruism; well-targeted science research should beat that (where did mosquito nets come from?).
The first time I read Torture vs. Specks about a year ago I didn't read a single comment because I assumed the article was making a point that simply multiplying can sometimes get you the wrong answer to a problem. I seem to have had a different "obvious answer" in mind.
And don't get me wrong, I generally agree with the idea that math can do better than moral intuition in deciding questions of ethics. Take this example from Eliezer’s post Circular Altruism which made me realize that I had assumed wrong:
I agree completely that you pick number 2. For me that was just manifestly obvious, of course the math trumps the feeling that you shouldn't gamble with people’s lives…but then we get to torture vs. dust specks and that just did not compute. So I've read most every argument I could find in favor of torture(there are a great deal and I might have missed something critical), but...while I totally understand the argument (I think) I'm still horrified that people would choose torture over dust specks.
I feel that the way that math predominates intuition begins to fall apart when you the problem compares trivial individual suffering with massive individual suffering, in a way very much analogous to the way in which Pascal’s Mugging stops working when you make the credibility really low but the threat really high. Like this. Except I find the answer to torture vs. dust specks to be much easier...
Let me give some examples to illustrate my point.
Can you imagine Harry killing Hermione because Voldemort threatened to plague all sentient life with one barely noticed dust speck each day for the rest of time? Can you imagine killing your own best friend/significant other/loved one to stop the powers of the Matrix from hitting 3^^^3 sentient beings with nearly inconsquential dust specks? Of course not. No. Snap decision.
Eliezer, would you seriously, given the choice by Alpha, the Alien superintelligence that always carries out its threats, give up all your work, and horribly torture some innocent person, all day for fifty years in the face of the threat of a 3^^^3 insignificant dust specks barely inconveniencing sentient beings? Or be tortured for fifty years to avoid the dust specks?
I realize that this is much more personally specific than the original question: but it is someone's loved one, someone's life. And if you wouldn't make the sacrifice what right do you have to say someone else should make it? I feel as though if you want to argue that torture for fifty years is better than 3^^^3 barely noticeable inconveniences you had better well be willing to make that sacrifice yourself.
And I can’t conceive of anyone actually sacrificing their life, or themselves to save the world from dust specks. Maybe I'm committing the typical mind fallacy in believing that no one is that ridiculously altruistic, but does anyone want an Artificial Intelligence that will potentially sacrifice them if it will deal with the universe’s dust speck problem or some equally widespread and trivial equivalent? I most certainly object to the creation of that AI. An AI that sacrifices me to save two others - I wouldn't like that, certainly, but I still think the AI should probably do it if it thinks their lives are of more value. But dust specks on the other hand....
This example made me immediately think that some sort of rule is needed to limit morality coming from math in the development of any AI program. When the problem reaches a certain low level of suffering and is multiplied it by an unreasonably large number it needs to take some kind of huge penalty because otherwise to an AI it would be vastly preferable the whole of Earth be blown up than 3^^^3 people suffer a mild slap to the face.
And really, I don’t think we want to create an Artificial Intelligence that would do that.
I’m mainly just concerned that some factor be incorporated into the design of any Artificial Intelligence that prevents it from murdering myself and others for trivial but widespread causes. Because that just sounds like a sci-fi book of how superintelligence could go horribly wrong.