That's a good point, and why I like Gary Drescher's handling of such dilemmas in Good and Real (ch. 7), which allows a kind of consequentialist but acausal (or extracausal) reasoning. His decision theory is the TDT-like "subjunctive reciprocity" which says that "You should do what you self-interestedly wish all similarly-situtated beings would do, because if you would regard it as optimal, so would they". That is, as a matter of the structure of reality, beings that are similar to you have similar input/output relationships -- so make sure you instantiate "good" I/O behavior.
The upshot is that subjunctive reciprocity can justify not harvesting in this case, and similar cases where we can assume no negative causal fallout from the doctor's decision. Why? Because if the doctor reasons that way (kill/harvest the patient "for the greater good") it logically follows that other people in similar situations would do the same thing (and thereby expect and adapt for it), even though there would be (by supposition) no causal influence between the doctor's decision and those other cases. On the contrary: like in Newcomb-like problems, the relevant effect is not one your decision causes; instead, your decision and the effect are common causal descendents of the same parent -- the parent being "the extent to which humans instantiate a decision theory like this".
Similarly, Drescher reasons e.g. that "I should not cheat business partners, even if I could secretly get away with it, because if I regarded this as optimal, it's more likely that my business partner regards it as optimal to do to me." More importantly, "If I make my decision depend on whether I'm likely to get caught, so will my partner's decision, a dependency I do not wish existed."
Consider the case of a 1000x run of Prisoner's Dilemma between 100 "similar" people, where they get randomly paired and play the game many times, but don't know anyone else's decision until it's all over. In that case, if you always cooperate, you are more likely to find yourself with more cooperating partners, even though your decisions can't possibly causally influence the others.
I discussed the merit of incorporating acausal consequences into your reasoning in Morality as Parfitian-Filtered Decision Theory.
As for whether it's a good model of human behavior to assume that people even think about "Does the doctor follow TDT?", look at it this way: people may not think about it in these exact terms, but then, why else are people convinced by arguments like, "What if everyone did that?" or "How would you like it if the roles were reversed?", even when
It seems they're relying on these "subjunctive acausal means-ends links" (SAMELs in my article) that cannot be justified in consequentialist-causal terms.
Summary: If you object to consequentialist ethical theories because you think they endorse horrible or catastrophic decisions, then you may instead be objecting to short-sighted utility functions or poor decision theories.
Recommended: Decision Theory Paradox: PD with Three Implies Chaos?
Related: The "Intuitions" Behind "Utilitarianism"
The simple idea that we ought to choose actions according to their probable consequences, ever since it was formulated, has garnered a rather shocking amount of dissent. Part of this may be due to causes other than philosophical objections, and some of the objections get into the metaphysics of metaethics. But there's a fair amount of opposition on rather simple grounds: that consequentialist reasoning appears to endorse bad decisions, either in the long run or as an effect of collective action.
Every so often, you'll hear someone offer a reductio ad absurdum of the following form: "Consider dilemma X. If we were consequentialists, then we would be forced to choose Y. But in the long run (or if widely adopted) the strategy of choosing Y leads to horrible consequence Z, and so consequentialism fails on its own terms."
There's something fishy about the argument when you lay it out like that: if it can be known that the strategy of choosing Y has horrible consequence Z, then why do we agree that consequentialists choose Y? In fact, there are two further unstated assumptions in every such argument I've heard, and it is those assumptions rather than consequentialism on which the absurdity really falls. But to discuss the assumptions, we need to delve into a bit of decision theory.
In my last post, I posed an apparent paradox: a case where it looked as if a simple rule could trump the most rational of decision theories in a fair fight. But there was a sleight of hand involved (which, to your credit, many of you spotted immediately). I judged Timeless Decision Theory on the basis of its long-term success, but each agent was stipulated to only care about its immediate children, not any further descendants! And indeed, the strategy of allowing free-riding defectors maximizes the number of an agent's immediate children, albeit at the price of hampering future generations by cluttering the field with defectors.1
If instead we let the TDT agents care about their distant descendants, then they'll crowd out the defectors by only cooperating when both other agents are TDT,2 and profit with a higher sustained growth rate once they form a supermajority. Not only do the TDTs with properly long-term decision theories beat out what I called DefectBots, but they get at least a fair fight against the carefully chosen simple algorithm I called CliqueBots. The paradox vanishes once you allow the agents to care about the long-term consequences of their choice.
Similarly, the purported reductios of consequentialism rely on the following two tricks: they implicitly assume that consequentialists must care only about the immediate consequences of an action, or they implicitly assume that consequentialists must be causal decision theorists.3
Let's consider one of the more famous examples, a dilemma posed by Judith Jarvis Thomson:
First, we can presume that the doctor cares about the welfare, not just of the five patients and the traveler, but of people more generally. If we drop the last supposition for a moment, it's clear that a consequentialist utilitarian doctor shouldn't kill the traveler for his organs; if word gets out that doctors do that sort of thing, then people will stay away from hospitals unless they're either exceptional altruists or at the edge of death, and this will result in people being less healthy overall.4
But what if the doctor is confident of keeping it a secret? Well, then causal decision theory would indeed tell her to harvest his organs, but TDT (and also UDT) would strongly advise her against it. Because if TDT endorsed the action, then other people would be able to deduce that TDT endorsed the action, and that (whether or not it had happened in any particular case) their lives would be in danger in any hospital run by a timeless decision theorist, and then we'd be in much the same boat. Therefore TDT calculates that the correct thing for TDT to output in order to maximize utility is "Don't kill the traveler,"5 and thus the doctor doesn't kill the traveler.
The question that a good consequentialist ought to be asking themselves is not "What happens in situation Y if I do X?", nor even "What happens in general if I do X whenever I'm in situation Y", but "What happens in general if everyone at least as smart as me deduces that I would do X whenever I'm in situation Y"? That, rather than the others, is the full exploration of the effects of choosing X in situation Y, and not coincidentally it's a colloquial version of Timeless Decision Theory. And as with Hofstadter's superrationality, TDT and UDT will avoid contributing to tragedies of the commons so long as enough people subscribe to them (or base their own decisions on the extrapolations of TDT and UDT).
In general, I'd like to offer (without proof) the following rationalist ethical inequality:
Your true valuation of all consequences + a good decision theory ≥ any particular deontology.
Now, a deontological rule might be easier to calculate, and work practically as well in the vast majority of circumstances (like approximating real physics with Newtonian mechanics). But if you have to deal with an edge case or something unfamiliar, you can get in trouble by persisting with the approximation; if you're programming a GPS, you need relativity. And as rule utilitarians can point out, you need to get your deontological rules from somewhere; if it's not from a careful consequentialist reckoning, then it might not be as trustworthy as it feels.6
Or it could be that particular deontological rules are much more reliable for running on corrupted hardware, and that no amount of caution will prevent people from shooting themselves in the foot if they're allowed to. That is a real concern, and it's beyond the scope of this post. But what's actually right probably doesn't include a component of making oneself stupid with regard to the actual circumstances in order to prevent other parts of one's mind from hijacking the decision. If we ever outgrow this hardware, we ought to leave the deontologies behind with it.
Footnotes:
1. Note that the evolutionary setup is necessary to the "paradox": if Omega dished out utils instead of children, then the short-term strategy is optimal in the long run too.
2. This is only right in a heuristic sense. If the agents suspect Omega will be ending the game soon, or they have too high a temporal discount rate, this won't work quite that way. Also, there's an entire gamut of other decision theories that TDT could include in its circle of cooperators. That's a good feature to have- the CliqueBots from the last post, by contrast, declare war on every other decision theory, and this costs them relative to TDT in a more mixed population (thanks to Jack for the example).
3. One more implicit assumption about consequentialism is the false dichotomy that consequentialists must choose either to be perfectly altruistic utilitarians or perfectly selfish hedonists, with no middle ground for caring about oneself and others to different positive degrees. Oddly enough, few people object to the deontological rules we've developed to avoid helping distant others without incurring guilt.
4. I'm assuming that in the world of the thought experiment, it's good for your health to see a doctor for check-ups and when you're ill. It's a different question whether that hypothetical holds in the real world. Also, while my reply is vulnerable to a least convenient possible world objection, I honestly have no idea how my moral intuitions should translate to a world where (say) people genuinely didn't mind knowing that doctors might do this as long as it maximized the lives saved.
5. The sort of epistemic advantage that would be necessary for TDT to conclude otherwise is implausible for a human being, and even in that case, there are decision theories like UDT that would refuse nonetheless (for the sake of other worlds where people suspected doctors of having such an epistemic advantage).
6. The reason that morality feels like deontology to us is an evolutionary one: if you haven't yet built an excellent consequentialist with a proper decision theory, then hard-coded rules are much more reliable than explicit reasoning.