Your argument is cogent, and yet I find the overwhelming majority of calls for diversity to be somehow underhanded. I suspect that your true motives are invisible to you. Consider this: is your motivation for valuing diversity really a product of your philosopher's thirst for pure, pristine knowledge, or do you just want every social group you see as important to be loaded with demographics which support your political faction? (Think carefully--the truth might not be obvious from casual introspection; we are masters at self-delusion when politics is at play.)

I say this because I cannot help but notice that the cry of "Diversity!" is invoked exclusively by those who are trying to import to a group those demographics which tend to offer political support to the left. What's more, the frequency which with this cry is invoked correlates positively with the degree to which that demographic supports the left. Consider the following data from the 2012 presidential election:

Whites voted 39% for Obama, and 59% for Romney. Blacks voted 93% for Obama, and 6% for Romney. Hispanics voted 71% for Obama, and 27% for Romney. Asians voted 73% for Obama, and 26% for Romney.

Source

When I encounter someone singing the praises of diversity, I more often find that they are lobbying for Blacks than Hispanics, rarely for Asians, and never for Whites. Blacks offer overwhelming support to the left, Hispanics are more lukewarm, Asians' support proportionally resembles that of Hispanics' (but they are a smaller group overall so it is less important for the left to signal respect for their faction), and Whites support the right. Coincidence? Unlikely.

Now consider gender (same source as above):

Men voted 45% for Obama, 52% for Romney. Women voted 55% for Obama, 44% for Romney.

Again, women support the left and men do not. Again, the cry of "Diversity!" is invoked for those trying to add women to a group, and rarely for men. I seem to encounter such arguments invoked as often for women as I do for racial minorities. While women do not favor the left as heavily as Hispanics or Blacks do, they are a larger group than all racial minorities combined, and so it is highly important for the left to signal respect for this demographic, and to ensure that they occupy positions of prestige and influence.

The overwhelming majority of people shouting, "Diversity!" are not motivated by epistemology at all. They are subconsciously (sometimes even consciously) making a power grab. That is all. You can tell by who, exactly, they are trying to include and in what they are trying to include them. For one, they are always lobbying for a demographic on the grounds that said demographic will bring additional knowledge to a discussion, but not for someone from a specific field of expertise which would be relevant to said discussion. There is likely to be more intellectual diversity between an exclusively middle class white male group comprising a physicist, a lawyer, a mathematician, a programmer, a chemist, a politician, an economist, and a businessman than there is between a demographically diverse group of eight people randomly selected from the general population. And you regularly see the pro-diversity crowd lobbying for their favored demographics to occupy positions in which being demographically distinct cannot possibly be an advantage, such as in the hard sciences. I find the champions of diversity disingenuous in the extreme.