You are ignoring the fact that the historical developments commonly known as "civil rights" have in fact led to a progression of ever more extreme policies (i.e. a slippery slope) whose present outcome is controversial even in the mainstream. This is indisputable no matter what position (if any) you happen to support in these controversies.

This slippery slope can be roughly described with the following progression:

1: Government-mandated discrimination across racial/ethnic/religious groups.

2: Libertarian/classical liberal position: procedural equality for everyone as far as the government is concerned, freedom to discriminate (or not) for private parties.

3: Prohibition of overt discrimination even for private businesses and organizations.

4(a): Affirmative action -- the government (and private parties under its influence and pressure) actively try to equalize statistical outcomes across groups by favoritism towards members of groups that do worse on average.

4(b): Disparate impact doctrine -- even if there is no overt discrimination, unequal statistical group outcomes are considered as evidence of discrimination by themselves, and any institution that produces such outcomes can be held legally liable on that basis alone.

While 1-3 are no longer controversial in the mainstream, 4(a) and 4(b) are still matters of intense public controversy. (Admittedly, for unclear reasons, 4(b) gets far less publicity than 4(a), despite its arguably even greater impact in practice.)