I should note this is well-travelled ground; we have this conversation about twice a year.

You are, of course, correct about the failure mode of the adjust-towards-preferred-score strategy.

The vote-without-reference-to-preferred-score strategy has the problem where, if comment C1 has a score 5x higher than C2, someone reading them can't tell whether that reflects 5x as many people reading C1 as C2 and endorsing it about as much, or a 5x higher proportion of readers endorsed it. Which is kind of a big deal, if we want karma scores to serve as feedback about comment quality/desirability rather than thread popularity, which some of us do.

People vary in terms of which problem they're more motivated to avoid, so you end up with both behaviors in the community. When the subject comes up, proponents of adjust-toward-preferred-score typically dismiss the concern you describe by positing that there's no systematic difference between early voters and late voters, and proponents of ignore-preferred-score dismiss the above concern by positing that there's not that much variation in how many people read any given pair of comments. (Personally, I suspect both of those statements are false.)