Theory off the top of my head: The causation is in the wrong direction. People who are rational are far more likely to be very systems-oriented, have limited social experiences as children (by having different interests and/or being too dang smart), be highly introverted, and other factors that correlate with being around other people a lot less than your typical person. There's nothing wrong with our hardware per se, it's just that we missed out on critical training data during the learning period,

Anecdotal: I have mild prosopagnosia. I have a lot of trouble recognising people outside their expected context, I make heavy use of non-facial cues. I'm pretty good at putting specific names to specific faces on demand when it feels important enough, although see prev point about expected context. I don't feel like I use anything resembling Bayesian reasoning, I feel like I have the same sense of recognition that I imagine most people have, it's just less dependent on seeing their face and more on other traits (most typically voice and manner of movement).