Not everybody who makes it to adulthood has kids. Various disabled people don't find a partner with whom to have kids.

I'm relatively confident that medical progress will be stronger than the effects you talk about in the next century. Sooner or later I expect genetic engineering to replace natural selection as the driving factor for human DNA, so the effects of the selection become less important.