Can't be bothered to dig up a reference, but boxing and American football at the very least are pretty unambiguously unhealthy: repeated concussions do nasty dementia-like things to your brain, and the incidence rates are surprisingly high (I think the phrase to google is "chronic traumatic encephalopathy"). I also wouldn't be surprised if the kind of joint problems that contact sports tend to give you led to mobility problems and attendant issues later in life, although that's a little more speculative.