utilitarianism isn't extreme altruism. It's just a way of trying to quantify morality. It doesn't decide what you care about. I'm pretty tired of people reacting to the concept of Utilitarianism with "Oh shit does that mean I need to give away all my money and live subsistence style to be a good person!?" A selfish utilitarian is just as possible as an extremely altruistic one or as one who's moderately altruistic.

There's enough ambiguity here that I'm not totally sure, but it sounds like you're describing consequential ethics, not utilitarianism as such. Utilitarianisms vary in details, but they all imply that people's utility is fungible, including that of their adherents; that a change in (happiness, fulfillment, preference satisfaction) is just as significant whether it applies to you or to, say, a bricklayer's son living in a malarial part of Burkina Faso.

It's certainly possible to claim utilitarian ethics and still prioritize your own utility in practice. But that's inconsistent -- aside from a few quibbles regarding asymmetric information -- with being a good person by that standard, if the standard means anything at all.