It has a big impact on the question of average IQ and ancestry, though. Brahmin average IQ is probably north of 100,* and so a first-rank mathematician coming from a Brahmin family of any wealth level is not as surprising as a first-rank mathematician coming from a Dalit family.

You'd need to examine the IQ of the poorer Brahmins, though, before you could say it's not surprising; otherwise if the poor Brahmins have the same IQs as equally poor Dalits, then it ought to be equally surprising.

One way to distinguish between the two is to evaluate the counterfactual statement "if they were there, they wouldn't be missed," and while Ramanujan is evidence for that statement it's weakened because of the potential impact of caste prejudice / barriers.

But Ramanujan is evidence against the Great Filters of nationality and poverty, which ought to be much bigger filters against possible Einsteins than caste.

It seems like the example of China might be better; it seems that young clever people have had the opportunity to escape sweatshops and cotton fields and enter the imperial service / university system for quite some time.

Yes, but I'm not very familiar with the background of major Chinese figures (eg. I just looked him up now and while I had assumed Confucius was a minor aristocrat, apparently he was actually the son of an army officer and "is said to have worked as a shepherd, cowherd, clerk, and a book-keeper."); plus, you'd want to look at the post-Tang major Chinese figures, but that will exclude most major Chinese figures period like all the major philosophers - looking up the Chinese philosophy table in Murray's Human Accomplishment, like the first 10 are all pre-examination (and Murray comments of one of them, " it was Zhu Xi who was responsible for making Mencius as well known as he is today, by including Mencius’s work as part of “The Four Books” that became the central texts for both primary education and the civil service examinations").