This is quite common argument, nevertheless I have never seen any detailed explanation of how having colonies could speed the developement. Comparison between different European countries suggests otherwise. For long time possessor of largest colonial empire, Spain, has briefly profited from colonial wealth in 16th century, a period followed by relatively slow developement leaving the country among the poorest in Europe in 19th century. There are many rich countries that never had any colonies.

And there is also quite standard argument that access to cheap labour or even slavery does more harm than good, because there is no motivation for inventions.