(I like your way of thinking, and I like even more that you look at this problem in the first place. I've had this mental note that says "utilitarianism vs guilt???" for a while now.)

One facet of the problem I think you overlooked is the "social group" dynamic.

Consider, which of these two is a more accurate expansion of "I'm a utilitarian" as observed in Real Life (TM):

• "My goal is to save lives effectively, in accordance with a coherent utility function (etc.)"

• "I think it's 'good'/'proper' to save lives effectively, in accordance with a coherent utility function (etc.)"