There would also be strong selective pressure for any genes which can override the segregation-distorter, even if the females can't recognize the males which carry it.

"Strong" for sure. Unfortunately for the species it would have to emerge fully functional in the time it takes for the species to evolve to extinction. Not so easy.