I'm not sure I agree with your view of colonialism.

What specific historical details do you contest?

Europeans did not uniformly judge all of the non-white peoples they encountered so it's not just a matter of ethnic chauvinism.

But they did pretty-uniformly judge the peoples they grouped into the category "black", which just to be clear is the group I specified and the group you're talking about too.

(1) there is a group of people in the world known as "blacks";

Who were originally grouped long ago, on the basis of the exceedingly superficial detail of skin color, a trait that turned out to be a red herring since they don't form a "natural group" in the sense that was assumed originally.

(2) there is a group of people in the world known as "whites";

See previous, with the added note that this level of grouping didn't take as thoroughly or as readily outside the colonies.

(3) there is a large an intractable difference in intelligence between these groups;

Disagreed. There is a large, thus-far intractable difference in performance on IQ tests between these groups; we do not concur as to what IQ tests are measuring, let alone the reasons for that.

and (4) it's reasonable to ask whether genetics might play a significant role in this gap.

But, given what we now know about the genetics of the groups in question, it's privileging the hypothesis to treat "blacks" as a natural group as opposed to a socially-constructed one, and given the many other plausible hypotheses not contradicted by evidence (and the data about historical power asymmetries in their interactions) it's hardly as primarily or all-consumingly interesting to focus on genetics, when there are so many other relevant factors that turn out not to be undermined by biology.

Just because the genetic evidence has come in does not mean that centuries of racism vanished overnight, and the idea of blacks as a natural group and the differences between them and whites as attributable to genetic factors are quite a bit older than our understanding of what genetics even was. It's no surprise they're still kicking around, influencing white intellectual types who've never personally been on the oppressed side of the equation and can't easily understand what all the fuss is about and why people might get so angry that they're still trying to talk about it in those terms...