That would account for reputation, not badges. (No one says "Hey, I got two answer from people with the same rep, but one has twice as many badges, so I'll go with that one.")

On the actual question, I've seen meta-posts on Stack Exchange complaining that they qualified for a badge and didn't get it, so the stuff does matter somewhat.