I disagree with that approach because this is exactly what is called being "subjective" by most people

Those same people are badly confused, because they usually believe that if ethical propositions are "subjective", it means that the choice between them is arbitrary. This is an incoherent belief. Ethical propositions don't become objective once you specify the agent's values; they were always objective, because we can't even think about an ethical proposition without reference to some set of values. Ethical propositions and values are logically glued together, like theorems and axioms.

You could say that the concept of something being subjective is itself a confusion, and that all propositions are objective.

That said, I share your disdain for philosophical word games. Personally, I think we should do away with words like 'moral' and 'good', and instead only talk about desires and their consequences.