As mentioned the environment plays a large part. My main point is, a more robust phenotype may have been selected for in harsher climates - thicker bones, larger skulls, larger brains. An example are Neanderthals - they became cold adapted, and therefore had a 'robust' phenotype. Robust hominins often have thickset skeletons and large heads - and therefore somewhat larger brains as a consequence.

The actual average size of the Neanderthal brain - was not significantly greater than modern humans. Also the Neanderthals had a different shaped skull - and a different version of a hominin brain. While the brain itself does not fossilize - endocasts can be taken from the interior of the skull.

This has been done by the Max Planck Institute - and the endocasts do not match our brain.

Additional research by the Max Planck Institute supports the thesis that the Neanderthal FOXP2 has undergone the same two amino acid changes as us - therefore may have some manner of complex speech??? Humans are not directly descended from the Neanderthal: they are a eurasian production - whereas we are african.

The common ancestor is taken to be Homo heildelbergensis - which was a large brained Hominin - in both African and Eurasian versions.

In all 3 cases - Heidelbergensis - Neanderthal - Homo sapiens - the Genomes will be very close.

Unfortuately none of the present Homo Heidelbergensis fossils are in good enough to be tested. The same goes for Homo erectus.