Yes, Shalizi was talking about something completely different, but his attitude was similar to yours. He was saying: "sure, I could imagine that it might be so (that there might be a heritable difference), but why are you so invested in believing in that? Why do you fight for it so much?". I meant for my quotation to bolster your case.

You are absolutely correct on the facts, and in a saner world I could leave it at that, but you seem to have missed an unspoken part of the argument; The common factor isn't genetics per se but rather an appeal to inherent nature. Whether that nature is the genetic legacy of selection for vastly different ancestral environments or due to the epigenetics of sexual dimorphism is very important in a scientific sense but not in the metaphysical sense of presenting a challenge to the ideals of "equality" or the "psychic unity of mankind." When Dr Shalizi writes the rhetorical question "why it is so important to you that IQ be heritable and unchangeable?" in the context of "'human equality' and 'genetic identity'" his tone is not that of scientific skepticism of an unproven claim but rather an apologetic defense of an embattled creed. Really, why is it so important to you what the truth is? After all, we don't have any evidence to suggest that the doctrines are wrong, so why not just repeat the cant like everyone else? Who else but a heretic would feel need to ask uncomfortable questions? For the most part, scientists writing against the hereditarian position don't bother debating the facts anymore; now that actual genetic evidence is starting to come out they know it'll just make them look foolish in a few years, and the psychometric evidence has survived four decades of concentrated attack already. It's all about implications and responsibility now, or in other words that the lie is too big to fail. It's hardly important to them if the truth at hand is a genetic or an hormonal inequality, they just want it to go away.