I find it incredible that any thoughtful person who thinks about these sources would conclude that they produce pure 'progress', without a taint of 'drift',

Who said anything about pure progress? A mixture of progress and drift is still a net improvement, unless there's something special about the starting point that makes drift a net negative.