I think what I'm looking for is some equivalent to Jaynes's "Desiderata" for probability, but in the realm of either basic utility functions or how to combine them.

The VNM theorem goes from certain hypotheses about your preferences to the existence of a utility function describing them. However, the utility function is defined only up to an affine transformation. This implies that given only that, there is no way to add up utilities, even the utilities of a single person. (You can, however , take weighted averages of them.) It also deals only with a single person, or rather, a single preference relation. It is silent on the subject of how to combine different people's preference relations or utility functions. There is no standard answer to the question of how to do this.

Being new to this, I'm also interested in a pointer to some kind of standard argument for (any kind of) utilitarianism.

You could try Peter Singer and the people who take that argument seriously.