Ok, that's a fair enough point. Sure, feelings do matter. However, I generally distinguish between genuine terminal preferences and mere surface emotions. The reason for this is that often it is easier/better to change your feelings than for other people to change their behaviour. For example, if I strongly dislike the name James Miller, you probably won't change your name to take my feelings into account.

(At the risk of saying something political: This is the same reason I don't like political correctness very much. I feel that it allows people to frame political discourse purely by being offended.)