Do Hermoine's parents even have the right to withdraw her? Harry's parents apparently do not have such a right:

Muggles had around the same legal standing as children or kittens: they were cute, so if you tortured them in public you could get arrested, but they weren’t people. Some reluctant provision had been made for recognizing the parents of Muggleborns as human in a limited sense, but Harry’s adoptive parents did not fall into that legal category (Chapter 26).