Who said anything about pure progress? A mixture of progress and drift is still a net improvement, unless there's something special about the starting point that makes drift a net negative.

Oh, I agree. But some people worry that if drift is not completely suppressed, over time it can completely reverse your goal structure