I would agree with your explanation.

Also, in the job example once you get to interview/test stage the observations should indeed clearly swamp out all priors based on what group the candidate belongs to. However earlier in the process (when sifting through thousands of similar resumes) could these priors still retain some importance?

Basically I would separate 2 types of discrimination:

• (1) I will not hire a person from group B because I don't like people from group B. Or I believe people from group B will almost certainly perform less well than people from group A.

• (2) I know the prior distribution of job performance for groups A and B (A is higher on average). After taking into account my obervations (looking at a resume) about 1 candidate from each group, the posterior distribution indicates that the candidate from group A is expected to perform better. So I hire A. Had I ignored the prior I would have hired B.

(1) is sub-optimal clearly unacceptable. (2) seems theoretically optimal and appears to be used for many groupings, like [went to a top university] vs. [medium university - same gpa/experience]

However (2) is completely unacceptable for other groupings (like race). Possible explanations:

• It has no impact anyway. For these groupings any differences in priors would be so tiny that they would immediately get overwhelmed by the slightest job application relevant info
• These are groupings for which people have absolutely no control. It is unfair that top group B people need to systematically overcome this prior.
• In practice no one will be able to apply this properly and everyone will end up amplifying priors and giving them way too much importance, so it is best to not go near it.