I guess if I were making the decision, I'd pick the undergraduate, unless the graduate transfer had some kind of story about what happened (a divorce, a death, something that would explain the failures).

Well, of course there's some kind of story that explains the failures, other than "he just wasn't good enough"; by assumption the guy wasn't let into the first graduate school off the street -- he had been a brilliant undergraduate. So something went wrong. The question is whether you should filter out a clearly capable person because something went wrong once. If you do, you're simply not optimizing for the right thing.

A rational deliberation on Candidate A would look like this: "This person obviously has issues, and he might not succeed here. But if he does, there's a good chance he'll succeed spectacularly -- at least, at a level above that of many students we're happy to call 'successful'. If he doesn't, well, some percentage of the blemish-free students we admit are going to fail anyway, so what's the difference?"

It is a fact of life that sometimes multiple iterations are required for something to work; but the school system does a poor job of accommodating this. Repeating a course as many times as necessary until you get an A is not something that the system encourages; if it were optimizing for the right things, it would be.

About humanities academia, this isn't at all true

Yes, that was a flippant, stereotyped exaggeration -- especially since "humanities" includes philosophy and linguistics, where the mentality is often very similar to math. Still, like any good caricature, it has some basis in reality. Generally speaking, the "softer" the subject -- the fewer objective measures of competence -- the greater the reliance on pure status games, i.e. academic snobbery. And of course my whole point here is that this snobbery is present even in "hard" fields.