Many people believe this physical fact should inform the way the kid is raised, including the kind of social roles they are prepared for.

Those beliefs are not really facts about the baby. Where did the beliefs come from? Isn't there a counter-factual history where (1) society works, (2) no one has those beliefs?

You should counter that "society works" just assumes the conclusion. I could respond by pointing to societies that actually existed in other places and times. And I'm sure that there are further responses on both sides.

Ultimately, the problem is that our differences in terminal values affect what sorts of things we consider good evidence. Or, if one were a moral realist, this would be evidence that one of us is so defective that they can't perceive moral truths.