I have a colleague who worked in the City for a few years (programming, not finance). The money was great, but eventually he looked around at the people he worked with, saw exactly what you've seen, and got out.

They have commitments to their families, a million a year feels like poverty after you're used to living on two million, and every day they add another thread to the ropes holding them in place.