Ummmmm... do I draw the line around the whole of the human race? I'm not sure whether I do or not. I do know that there is a certain boundary (defined mostly by culture) where I get much more likely to say 'that's your problem' and become much less skeptical/cynical about preferences, although issues that seem truly serious always get the same treatment.

For some reason, choosing to accept that somebody's utility function might be very different from your own feels kind of like abandoning them from the inside. (Subjective!).