MugaSofer comments on White Lies - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (893)
What do you mean by "consequentialist thought experiment"?
Yes, you can always argue that any behavior is instrumental, replacing it with the reason it came to be thought of as moral, but if you go down that route, you'll end up concluding the purpose of life is to maximize inclusive genetic fitness.
One of the standard thought experiments used to demonstrate and/or explain consequentialism. I'm really just trying to see what your model of consequentialism is based on.
Well, we're adaptation-executors, not fitness-maximizers - the environment has changed. But yeah, there's a very real danger in coming up with grandiose rationalizations for how all your moral intuitions are really consequences of your beautifully simple unified theory.
And there's a very real danger of this being a fully general counterargument against any sufficiently simple moral theory.
You're absolutely right about that. In fact, there's a danger that it can be a fully general counterargument against any moral theory at all! After all, they might simply be rationalizing away the flaws...
I wouldn't endorse using it as a counterargument at all, honestly. If you can point out actual rationalizations, that's one thing, but merely calling someone a sophisticated arguer is absolutely a Bad Idea.
Well, as Eliezer explained here, simple moral systems are in fact likely to be wrong.
I think that's one of the areas where Eliezer got it completely wrong. Value isn't that complex, and it's a mistake to take people's apparent values at face value as he seems to.
Our values are psychological drives from a time in our evolutionary history before we could possibly be consequentialist enough to translate a simple underlying value into all the actions required to satisfy it. Which means that evolution had to bake in the "break this down into subgoals" operation, leaving us with the subgoals as our actual values. Lots of different things are useful for reproduction, so we value lots of different things. I would not have found that wiki article convincing either back when I believed as you believe, but have you read "Thou art godshatter?"
People have drives to value different things, but a drive to value is not the same thing as a value. For example, people have an in-group bias (tribalism), but that doesn't mean that it's an actual value.
If values are not drives (Note I am saying values are drives, not "driives are values", "drives to value are values", or anything else besides "values are drives"), what functional role do they play in the brain? What selection pressure built them into us? Or are they spandrels? If this role is not "things that motivate us to choose one action over another," why are they motivating you to choose one action over another? If that is their role, you are using a weird definition of "drive", so define "Fhqwhgads" as "things that motivate us to choose one action over another", and substitute that in place of "value" in my last argument.
If values are drives, but not all drives are values, then... (a) if a value is a drive you reflectively endorse and a drive you reflectively endorse is a value, then why would we evolve to reflectively endorse only one of our evolved values? (b) otherwise, why would either you or I care about what our "values" are?
I agree that values are drives, but not all drives are values. I dispute that we would reflectively endorse more than one of our evolved drives as values. Most people aren't in a reflective equilibrium, so they appear to have multiple terminal values - but that is only because they aren't' in a reflective equilibrium.
What manner of reflection process is it that eliminates terminal values until you only have one left? Not the one that I use (At least, not anymore, since I have reflected on my reflection process). A linear combination (or even a nonlinear combination) of terminal values can fit in exactly the same spot that a single value could in a utility function. You could even give that combination a name, like "goodness", and call it a single value (though it would be a complex one). So there is nothing inconsistent about having several separate values.
Let me hazard a guess, based on my own previous reflection process, now abandoned due to meta-reflection. First, I would find a pair of thought experiments where I had strong feelings for an object-level choice in each, and I felt I was being inconsistent between them. Of course, object-level choices in two different scenarios can't be inconsistent. There is a computation that returns both of those answers, namely, whatever was going on in your pre-reflection brain.
For example, "throw the level, redirect the trolley to kill 1 instead of 5" and "don't butcher the healthy patient and steal their organs to save five."
The inconsistence is in the two principles I would have automatically come up with to explain two different object-level choices. Or, if my reasons for one emotional reaction are too complicated for me to realize, then it's between one principle and the emotional reaction. Of course, the force behind the principle comes from the emotional reaction to the thought experiment which motivated it.
Then, I would let the two emotions clash against each other, letting my mind flip between the two scenarios back and forth until one started to weaken. The winner would become stronger, because it survived a clash. And so did the principle my mind coughed up to explain it.
What are the problems with this?
It favors simple principles for the sole reason that they are easier to guess by my conscious mind, which of course doesn't really have access to the underlying reasons. It just thinks it does. This means it depends on my ignorance of other more complicated principles. This part can be destroyed by the truth.
The strength of the emotion for the object-level choice is often lent to the principle by something besides what you think it is. Yvain covered this in an essay that you, being a hedonistic utilitarian would probably like: Wirehead Gods on Lotus Thrones. His example is that being inactive and incredibly happy without interuption forever sounds good to him if he thinks of Buddhists sitting on lotuses and being happy, but bad if he thinks of junkies sticking needles in their arms and being happy. With this kind of reflection, you consciously think something like: "Of course, sitting on the lotus isn't inherently valuable, and needles in arms aren't inherently disvaluable either," but unconsciously, your emotional reaction to that is what's determining which explicit principles like "wireheading is good" or "wireheading is bad" you consciously endorse.
All of your standard biases are at play in generating the emotional reactions in the first place. Scope insensitivity, status quo bias, commitment bias, etc.
This reflection process can go down different paths depending on the order that thought experiments are encountered. If you get the "throw switch, redirect trolley" one first, and then are told you are a consequentialist, and that there are other people who don't throw the switch because then they are personally killing someone, and you think about their thought process and reject it as a bad principle, and then you see the "push the fat man off the bridge" one, and you think "wow, this really feels like I shouldn't push him off the bridge, but [I have this principle established where I act to save the most lives, not to keep my hands clean]", and slowly your instinct (like mine did) starts to become "push the fat man off the bridge." And then you hear the transplant version, and you become a little more consequentialist. And so on. It would be completely different if most people heard the transplant one first (or an even more deontology-skewed thought experiment). I am glad of course, that I have gone down this path as far as I have. Being a consequentialist has good consequences, and I like that! But my past self might not have agreed, and likewise I probably won't agree with most possible changes to my values. Each version of me judges differences between the versions under its own standards.
There's the so called sacred vs. secular value divide (I actually think it's more of a hierarchy, with several layers of increasing sacredness, each of which feels like it should lexically override the last), where pitting a secular value vs a sacred value makes the secular value weaker and the sacred one stronger. But which values are secular or sacred is largely a function of what your peers value.
And whether a value becomes stronger or weaker through this process depends largely on which pairs of thought experiments you happen to think of. Is a particular value, say "artistic expression", being compared to the value of life, and therefore growing weaker, or is it being compared to the value of not being offended, and therefore growing stronger?
So that you don't ignore my question like you did the one in the last post, I'll reiterate it. (And I'll add some other questions). What process of reflection are you using that you think leads people toward a single value? Does it avoid the problems with my old one that I described? Is this a process of reflection most people would meta-reflectively endorse over alternative ones that don't shrink them down to one value? (If you are saying that people who have several values are out of reflective equilibrium, then you'd better argue for this point.)
Edited: formatting.
So what, on your view, is the simple thing that humans actually value?
Establishing a lower bound on the complexity of a moral theory that has all the features we want seems like a reasonable thing to do. I don't think the connotations of "fully general counterargument" are appropriate here. "Fully general" means you can apply it against a theory without really looking at the details of the theory. If you have to establish that the theory is sufficiently simple before applying the counterargument, you are referencing the details of the theory in a way that differentiates from other theories, and the counterargument is not "fully general".
"This theory is too simple" is something that can be argued against almost any theory you disagree with. That's why it's fully general.
No, it isn't: Anyone familiar with the linguistic havoc sociological theory of systems deigns to inflict on its victims will assure you of that!