And traditional behavior gives us an imperfect window into the economics of the past, which is what's under discussion when we talk about historical selective fitness.
Traditional behavior is so widely varied, though, that it's difficult to draw any conclusions. Some traditional societies practiced polyandry, others, polygamy, and still others, levirate marriage, and avunculism, and so forth. Some traditional societies were accepting of homosexuality and even transgenderism. You say that cultures that prohibit childbirth die out, but many diverse cultures have a thriving tradition of monasticism (which is even worse for reproductive fitness than homosexuality!)
If gay sons were helpful enough with nephews and nieces that it was as if they had had their own children, it seems to me they would be welcomed and lauded as examples of loving selflessness.
Would they? "Gay" is a recent category; traditional societies did not attempt to classify humans in that way and it only became popular when religious authorities attempted to criminalize it and early psychologists attempted to medicalize it. Men were not "gay" or "homosexual", they were more or less inclined towards other men.
Epistemic status: speculating about things I'm not familiar with; hoping to be educated in the comments. This post is a question, not an answer.
ETA: this comment thread seems to be leading towards the best answer so far.
There's a question I've seen many times, most recently in Scott Alexander's recent links thread. This latest variant goes like this:
Obligate male homosexuality greatly harms reproductive fitness. And so, the argument goes, there must be some other selection pressure, one great enough to overcome the drastic effect of not having any children. The comments on that post list several other proposed answers, all of them suggesting a tradeoff vs. a benefit elsewhere: for instance, that it pays to have some proportion of gay men who invest their resources in their nieces and nephews instead of their own children.
But how do we know if this is a valid question - if the situation really needs to be explained at all?
For obvious political and social reasons, it's hard to be sure how many people are homosexual. Note that we are interested only in obligate homosexuality - bisexuals presumably don't have strongly reduced fitness. The Wikipedia article doesn't really distinguish obligate homosexuality from bi-, pan- and even trans-sexuals. The discussion in the SSC comments used an (unsourced?) range of 1%-3%, which seems at least consistent with other sources, so let's run with that.
The rate of major birth defects in the US, as reported by the CDC, is also about 3%. This counts both developmental and genetic problems, and includes everything from anencephaly (invariably fatal) through Down syndrome (severe but survivable) to cleft palates (minor). But most of these, at least 1.5% of births, were always fatal before modern medicine, and many of the others reduced fitness (via mate selection, if nothing else). Various other defects and diseases, which only manifest later in life, are also thought to be influenced or determined during early development. And so is sexual preference.
(Whether homosexuality is a developmental disorder is not the point; I'm comparing the effect of selection pressure on fatal teratology with its effect on reduced-fitness homosexuality.)
Embryological development is a complex and fragile process, and there are many ways for it to go wrong. We don't wonder how it is possible that selection pressure allows anencephaly to occur in 1 in 4859 births. There are certainly direct causes of anencephaly, explanations of why it happens when it does, but (I think) we don't a priori expect them to be due to tradeoffs yielding benefits elsewhere. It's just as plausible that the tradeoffs involved are against even worse (counterfactual) problems elsewhere - or that there are just no available mutations that don't have these or equally severe problems.
Could it be that linking sexual preference to the biological gender is, for some complex developmental reason, fragile enough that it goes wrong despite all selection pressure to the contrary, that it has no redeeming qualities from the viewpoint of evolution, and that is all there is to it?
When faced with any phenotype with reduced fitness, how can we judge if there is something to be explained - a beneficial tradeoff elsewhere to search for - or merely a hard problem evolution couldn't solve completely? And is there a way to quantify this question, relating it to the known mathematical models of genetics?
Notes:
1. I'm posting this in the spirit of recent suggestions to post more and accept lower quality of (our own) posts to Discussion.
2. I'm going to sleep now and will start replying to comments about 10 hours from now; sorry for the inconvenience.