That was my thought exactly. Any Dark (or Light) Lord rising to power would want to knock off any ancient immortal wizards who still happened to be around first, as they'd be an obvious threat to their plans. Historically immortals probably got wiped at the start of each big Wizarding War.