If Harry's theory is right, squibs can't be normal genetic descendants (mutation not withstanding) of wizards, but adultery is a very real, very common thing. Cannon does not rule out the possibility, though given that the books were meant to be accessible to children it's not surprising that Rowling doesn't go into detail on the matter.