I agree about gamma, cosine, and pi. I'm not troubled by the minus sign in the zeta function but suspect we should really be working with the related "xi function" whose symmetries are simpler. I'm not a very expert physicist but my guess is that the 4pi there is going to pop in one place or another and it doesn't matter very much which you choose.

The only one of these that I actually get cross about is the gamma function. With all the others, there are tradeoffs -- e.g., if you work with tau = 2pi instead of with pi, some things become simpler, some become more complicated, and on balance it's probably a slight improvement. If you work with the factorial function instead of the gamma function, I think pretty much every formula I've ever seen that uses it becomes simpler (usually by the omission of an annoying "-1" term).

(But I'm not an analytic number theorist or a complex analyst -- though I was kinda-sorta a bit of a complex analyst once -- and it's possible that the cognoscenti know of good reasons why gamma should stay the way it is.)

I would say cos is simpler than sin because its Taylor series has a factor of x knocked off.

In practice they tend to show up together, though. Often you can replace the pair with something like e^(i x), so maybe that should be considered the simplest.