Indeed, I doubt that everyone 1000 years ago is an ancestor of everyone living today. I expect that everyone 1000 years ago is an ancestor of everyone [Edit: at least within a geographical region], of no-one, or is atypical in some way (for example, I expect a family that is well-off to have a number of children sampled from a different distribution, which has no reason to have mean greater than 2).

You are right, though, that across the board N has to be greater than 2 or else the global population would shrink over time. Moreover, if (when we look at Slytherin's descendants specifically) N is 1 or less, we expect the Slytherin line to eventually die out. This leaves room for a line that neither dies out nor grows as quickly as population does overall.