If this wasn't LW, I would simply state the obvious an say that all sorts of evil stuff can be justified by replaceability.
I'm not sure how to parse this. One possible interpretation is, "If the replaceability thesis were true, then it would follow that people should do evil things. But since people shouldn't do evil things, it follows by modus tollens that the replaceability thesis is false." This kind of argument could be correct depending on how the details were fleshed out, but I certainly would not call it obvious.
Another interpretation is, "Unscrupulous clever arguers could use the replaceability thesis to persuade people to do evil things." This is more obvious, but it doesn't seem very relevant; sufficiently bad reasoning can be used to argue for any conclusion from any set of premises.
I wasn't trying to say anything deep, really. If the replaceability argument works for investment bankers, then it works for henchmen of an oppressive regime, too. In my country, many people actually used the replaceability argument, without the fancy name. And in hindsight people in my county agree that they shouldn't have used the argument. So yeah, maybe it's the modus tollens. But maybe it's simpler than that: maybe these people misjudged being completely replaceable. In the eighties more and more people dared to say no to the Hungarian secret service, with less and less consequences.
By the way, the apparently yet-unpublished part 2 of jkaufman's link will deal with this issue.
The current issue of the Oxford Left Review has a debate between socialist Pete Mills and two 80,000 hours people, Ben Todd and Sebastian Farquhar: The Ethical Careers Debate, p4-9. I'm interested in it because I want to understand why people object to the ideas of 80,000 hours. A paraphrasing:
As a socialist, Mills really doesn't like the argument that the best way to help the world's poor is probably to work in heavily capitalist industries. He seems to be avoiding engaging with Todd and Farquhar's arguments, especially replaceability. He also really doesn't like looking at things in terms of numbers, I think because numbers suggest certainty. When I calculate that in 50 years of giving away $40K a year you save 1000 lives at $2K each, that's not saying the number is exactly 1000. It's saying 1000 is my best guess, and unless I can come up with a better guess it's the estimate I should use when choosing between this career path and other ones. He also doesn't seem to understand prediction and probability: "every revolution is impossible, until it is inevitable" may be how it feels for those living under an oppressive regime but it's not our best probability estimate. [1]
In a previous discussion a friend also was mislead calculations. When I said "one can avert infant deaths for about $500 each" their response was "What do they do with the 500 dollars? That doesn't seem to make sense. Do they give the infant a $500 anti-death pill? How do you know it really takes a constant stream of $500 for each infant?". Have other people run into this? Bad calculations also tend to be distributed widely, with people saying things like "one pint of blood can save up to three lives" when the expected marginal lives saved is actually tiny. Maybe we should focus less on estimates of effectiveness in smart-giving advocacy? Is there a way to show the huge difference in effect between the best charities and most charities without using these?
Maybe I should have way more of these discussions, enough that I can collect statistics on what arguments and examples work and which don't.
(I also posted this on my blog)
[1] Which is not to say you can't have big jumps in probability estimates. I could put the chance of revolution at 5% somewhere based on historical data but then hear some new information about how one has just started and sounds really promising which bumps my estimate up to 70%. But expected value calculations for jobs can work with numbers like these, it's just "impossible" and "inevitable" that break estimates.