I'm not sure about the ancestral environment explanation. On one hand, you're relatively stuck with the people in your tribe, but on the other hand, they can't afford to throw away useful people.

I haven't heard anything about how status and harassment work in modern hunter-gatherer tribes, so I don't know whether the costs of mistakes are higher there or in modern high-tech societies.

One clue might come from the various situations of deaf people. There are (were?) a couple of isolated communities where 1 in 6 people were deaf. Everyone knew sign language, and hearing or deafness wasn't considered an important feature.

Another clue comes from social groups and families. It can take a lot for an obnoxious person to be pressured or ostracized-- it's socially cheaper to keep them around than to deal with their difficult habits.

I believe that one aspect of modern wealth is that it's more feasible to throw people away by various means of exclusion-- there are plenty of more people available immediately. This wouldn't have been true for hunter-gatherers.