One related problem is that even when one doesn't sound like Charles Murray, opponents will try to spin things to sound like they are racist claims. For example, try arguing that cultural attitudes matter for how much kids are willing to learn, and then see how long it takes to get accused of racism.
Frankly, when I mentioned Murray, I didn't even have in mind these most controversial and incendiary topics he's ever written on. (This was in my opinion a mistake on his part, since his association with them now automatically detracts from other interesting stuff about which he's written much more.) As you note, you don't have to go anywhere as far to get to ideas that will be unacceptable to the modern mainstream ideological consensus about issues of education, which will provoke angry denunciations that deny any legitimacy to your position instead of rational debate.
I want to learn what's well-understood about education. I expect to launch myself into some endeavors in teaching the first few levels of epistemic and instrumental rationality - ie., critical thinking and problem solving. I'm a little suspicious, though, of the scattered educational texts that I've so far read. In particular, education seems like a field where it's easy to have motivated thoughts, and hard to gather good data.
With my background (Math and CS) I'm a little at sea in educational literature. Does anyone know of good, reductionist-grade or evidential-grade, introductory texts in education?