Also, we cannot completely ignore the possibility of the "magic machinery" (the one that recognize the genetic marker) to have some kind of shuffling process that'll occasionally turn on or off the magical marker when an egg is fertilized. Either randomly, or based on events (triggers like "an egg fertilized exactly at the second where the moon is the fullest will have a high probability of having the magical marked added").

Or that there is no genetic marker at all and the machinery uses some algorithm of its own to determine who should have magic which is heavily biased towards children of wizards.