Can't be bothered to dig up a reference, but boxing and American football at the very least are pretty unambiguously unhealthy: repeated concussions do nasty dementia-like things to your brain, and the incidence rates are surprisingly high (I think the phrase to google is "chronic traumatic encephalopathy"). I also wouldn't be surprised if the kind of joint problems that contact sports tend to give you led to mobility problems and attendant issues later in life, although that's a little more speculative.
The question is - am I doing enough exercise?
I intend to provide a worked example for you to work alongside with your own calculations and decide if you should increase or decrease your exercise.
The benefits of physical activity are various and this calculation can be done for one or all of them; some of them include: