Have you read "A Human's Guide to Words" from the sequences? I recommend it highly, and I think it clears up our disagreement.
If we were computers, or perfect philosophers, you would be completely correct. As it stands, you're still mostly correct. Unfortunately, humans have certain defects which require us to take certain precautions, including caring about connotations as well as denotations, as also history of usage and so on.
Yes, I've read the post you mention. But since you didn't fully spell it out, I'm not sure what you're intending to say. I can think of at least two interpretations, one I would disagree with, and one I would agree with.
1) You might be saying that it is impossible to do what I am suggesting given the way people actually relate to words. In this case I would disagree, and I also suspect that you would not apply such a principle consistently, e.g. to speaking about the gender of trans people.
2) You might simply be saying that doing what I am proposing has an...