It usually makes more sense to try to convince observers than to convince the people you're debating with. It has a much higher rate of success. Even if the observers aren't really impartial, they're under less pressure to uphold their position.

Although it can be emotionally taxing to walk away, one of the primary factors I take into consideration when deciding whether it's worth having a debate at all is who else is listening to it.