That there are population level differences in IQ is not controversial (except in the sense that evolution is controversial because more than 30% of Americans don't believe in it).

That IQ is a useful proxy for general intelligence and a useful tool in determining life outcomes is not controversial.

That IQ is heritable is not controversial.

That the differences are genetic is controversial but the data does seem to suggest that much of the difference is indeed genetic and another portion is biological (pre-natal care, early nutrition).

http://occidentalascent.wordpress.com/2012/06/10/the-facts-that-need-to-be-explained/

Every objection you've listed so far has been addressed in exhaustive detail in the above link.