Okay, but if not everyone graduates from college, and the point of admissions is to weed out people who'll succeed in school rather than wasting everyone's time, then how does a college degree mean anything different for a standard graduate, a legacy graduate, and an affirmative-action graduate?

In the extreme case where being above a given level of competence deterministically causes graduation, you're correct and AA makes no difference; the likelihood (but not necessarily the prior or posterior probability) of different competence levels for a college graduate is independent of race. In the extreme case where graduation is completely random, you're wrong and AA affects the evidence provided by graduation in the same way as it affects the evidence provided by admission. Reality is likely to be somewhere in between (I'm not saying it's in the middle).

(Note that the bar is lowered for legacy graduates to the same degree as affirmative-action graduates, so if you don't hear "my father also went here" the same way as "I got in partly because of my race", then there's a different factor at work here.)

It depends on the actual distribution of legacy and AA graduates.