I live by my morals, whether or not others share them. This doesn't change the fact that they are my morals, a feature of my brain state, and not some sort of objective independently existing morals. It's exactly the same situation as saying that I feel that my girlfriend is beautiful whether or not others agree, but that doesn't mean there's an objective standard of beauty in the universe that doesn't depend on observers.

If I belonged to a low status group that most people had no moral issues with abusing, then I would keep saying they behave immorally according to my views, and they would keep ignoring my words and abusing me. I fail to see what about this situation suggests that I behave as if I believe in realist morals.