A moral theory that doesn't tell you how to act in daily life seems incomplete, at least in comparison to e.g. deontological approaches. If one defines a moral framework as something that does tell you how to act in daily life, as I suspect many of the people you're thinking of do, then to the extent that utilitarianism is a moral framework, it requires extreme self-sacrifice (because the only, or at least most obvious, way to interpret utilitarianism as something that does tell you how to act in daily life is to interpret it as saying that you are required to act in the way that maximizes utility).

So on some level it's just an argument about definitions, but there is a real point: either utilitarianism requires this extreme self-sacrifice, or it is something substantially less useful in daily life than deontology or virtue ethics.