Outliers are interesting, but I'm not sure they are often useful examples. I suspect the focus on outliers is more due to a certain insecurity among specialists, which is exactly the last thing 99.9% of the people struggling to understand or enjoy mathematics need further exposure to.

Perhaps within mathematics, progress really is so dominated by the elite that it seems natural to worry so much about elites. I don't know either way. But in most other fields, and in the everyday strength of society, there seems to be a decent potential from moving everyone else just a few notches in mathematical comfort.

Naturally, people rise to the level of their ability for any given level of pedagogic incompetence, and so it would be equally useless to blame people for not figuring out on their own how to maximize their own ability (whatever that may be), unless we can provide reasonably concrete advice.