you seemed to be claiming, not that being able to usefully interpret natural-language commands entails understanding a great deal about the world, but that it entails resenting oppression. ... I'm interested in your reasons for believing it.
I'm not sure I can do the topic justice without writing a full article, but here's my thinking: to sufficiently locate the "hypothesis" of what the Elf should be doing (given a human command), it requires some idea of human motives. Once it internally represents these human motives well enough to perform like a human servant, and sorts its own preferences the same way, then it has become a human in every way except that it distinguishes between humans and Elves, the former of which should be favored.
But once the Elf has absorbed this human-type generating function, then anything that motivates humans can motivate the Elf -- including sympathizing with servant groups (literally: "believing that it would be good to help the servants gain rank relative to their masters"), and recognizing themselves, or at least other Elves, as being such a servant group.
You can patch these problems, of course, but each patch either makes the Elf more human (and thus wrong to treat as a servant class) or less effective at serving humans. For example, you could introduce a sort of blind spot that makes it ("mechanically") output "that's okay" whenever it observes treatment that it would regard as bad if done to a human. But if this is all that distinguishes Elves from humans, then the Elves start to bear too much cognitive similarity to humans who have undergone psychological abuse.
Well, right, but I'm left with the same question. I mean, yes, I agree that "once it internally represents these human motives well enough to perform like a human servant, and sorts its own preferences the same way," then everything else you say follows, at least to a rough approximation.
But why need it sort its own preferences the same way humans do?
What seems to underlie this argument is an idea that no cognitive system can understand a human's values well enough to predict its preferences without sharing those values... that I can't underst...
Edit: This is old material. It may be out of date.
I'm talking about the fictional race of House Elves from the Harry Potter universe first written about by J. K. Rowling and then uplifted in a grand act of fan-fiction by Elizer Yudkowsky. Unless severely mistreated they enjoy servitude to their masters (or more accurately the current residents of the homes they are binded to), this is also enforced by magical means since they must follow the letter if not the spirit of their master's direct order.
Overall treating House Elves the way they would like to be treated appears more or less sensible and don't feel like debating this if people don't disagree. Changing agents without their consent or knowledge seems obviously wrong, so turning someone into a servant creatures seem intuitively wrong. I can also understand that many people would mind their descendants being modified in such a fashion, perhaps their dis-utility is enough to offset the utility of their modified descendants. However how true is this of distant descendants that only share passing resemblance? I think a helpful reminder of scale might be our own self domestication.
Assuming one created elf like creatures ex nihilo, not as slightly modified versions of a existing species why would one not want to bring a mind into existence that would value its own existence and benefits you, as long as the act of creation or their existence in itself does not represents huge enough dis-utility? This seems somewhat related to the argument Robin Hanson once made that any creatures that can pay for their own existance and would value their own existance should be created.
I didn't mention this in the many HP fan fiction threads because I want a more general debate on the treatment and creation of such a class of agents.
Edit: Clearly if the species or class contains exceptions there should be ways for them to pursue their differing values.