They wouldn't. Nobody ever acts other than by their own preferences. Me by mine, they by theirs. This is pretty much baked into the definition of 'preferences', although with non-utility-maximizers like humans the situation is more complex than we'd like.
This is inherent in your own description of the scenario. You said they abuse me. So presumably their preferences (including their morals) are OK with that.
I'm sure you understand all this. What made you think I believed anything different?
Prediction: If you were forced to consider the situation in near mode, e.g., if you had something to protect that was being threatened, you wouldn't be arguing that preferences are relative to the individual, but why the other person was acting amorally.
Here's the new thread for posting quotes, with the usual rules: