Are you asking rhetorically?

Sorry, I meant the two questions in different senses, I should have made that clearer.

The American legal justification for the disparate impact doctrine, (..) is the 1964 Civil Rights Act,

The Civil Rights Acts didn't specify disparate impact as opposed to disparate treatment.

and the legislative justification for that was a history of massive mistreatment of individuals based on skin color.

I understand the motivation, but I don't think the ever increasing (and rather arbitrary) list of protected groups is a workable approach. Not to mention the "some groups are more equal than others" problem implicit in having a specific list of "protected groups".

That does not speak to any philosophical objections you have but, frankly, no philosophical objections you make have any bearing on the legal justification.

If you look at the history of law, philosophical arguments end up influencing legal arguments all the time.