Very good point that I think clarified this for me.
Per Wikipedia, "Altruism or selflessness is the principle or practice of concern for the welfare of others." That seems like a plausible definition, and I think it illustrates what's wrong with this whole chain. The issue here is not increasing concern or practice but expanding the definition of "others"; that is, bringing more people/animals/objects into the realm of concern. So if we taboo altruism, the question becomes to whom/what and to what degree should we practice concern. Furthermore, on what grounds should we do this?
For instance, if the real principle is to increase pleasure and avoid pain, then we should have concern for humans and higher animals, but not care about viruses, plants, or rocks. (I'm not saying that's the right fundamental principle; just an example that makes it clearer where to draw the line.)
In other words, altruism is not a good in itself. It needs a grounding in something else. If the grounding principle were something like "Increase the status and success of my tribe", then altruistic behavior could be very negative for other tribes.
My main objection to Coherent Extrapolated Volition (CEV) is the "Extrapolated" part. I don't see any reason to trust the extrapolated volition of humanity - but this isn't just for self centred reasons. I don't see any reason to trust my own extrapolated volition. I think it's perfectly possible that my extrapolated volition would follow some scenario like this:
There are many other ways this could go, maybe ending up as a negative utilitarian or completely indifferent, but that's enough to give the flavour. You might trust the person you want to be, to do the right things. But you can't trust them to want to be the right person - especially several levels in (compare with the argument in this post, and my very old chaining god idea). I'm not claiming that such a value drift is inevitable, just that it's possible - and so I'd want my initial values to dominate when there is a large conflict.
Nor do I give Armstrong 7's values any credit for having originated from mine. Under torture, I'm pretty sure I could be made to accept any system of values whatsoever; there are other ways that would provably alter my values, so I don't see any reason to privilege Armstrong 7's values in this way.
"But," says the objecting strawman, "this is completely different! Armstrong 7's values are the ones that you would reach by following the path you would want to follow anyway! That's where you would get to, if you started out wanting to be more altruistic, had control over you own motivational structure, and grew and learnt and knew more!"
"Thanks for pointing that out," I respond, "now that I know where that ends up, I must make sure to change the path I would want to follow! I'm not sure whether I shouldn't be more altruistic, or avoid touching my motivational structure, or not want to grow or learn or know more. Those all sound pretty good, but if they end up at Armstrong 7, something's going to have to give."