People have been proposing that some people should be harmed to benefit others long before anyone proposed the idea of utilitarianism; usually it was justified because the people being harmed are outgroup members, or simply Less Important compared to the people being helped, or to subordinate individual identity to the group identity.

Sometimes, this doesn't go well. In some cases, such as, by your own example, in the case of taxes or a justic system, it goes much better than a refusal to harm some people to help others. Utilitarianism is a construction for formalizing under what conditions it is or is not a good idea to attempt such tradeoffs. Calling the purges of Hitler or Stalin failings of Utilitarianism is about as fair as calling every successful government intervention or institution ever, which after all all involve sacrificing resources of the public for a common good, successes of Utilitarianism.