This brings to mind the notion of Heterozygote advantage for certain traits. For example, there is the sickle-cell trait. One allele makes you highly resistant to malaria. Two gives you sickle-cell anemia. In a population where malaria is a grave threat, the trait is worth it in the general population even if some poor saps get shafted with the recessive genes. For reference, Wiki quotes a rate of 2% of Nigerian newborns having sickle-cell anemia.

If there's some process where homosexuality is a fail mode, then as long as it confers a net overall advantage one would expect it to persist.