Why do we think why would we want to signal niceness in the far view? And why should signaling "niceness" involve supporting the underdog?

If you think that the stronger side is morally superior, your decision is trivial, and doesn't signal anything. If one side is stronger while the other is morally superior, then supporting the strong side would be choosing individual best interest over communal best interest, which is like defecting in the prisoner's dilemma.