To expand on shminux's point about what has happened in the last 100 years that's different: There's a serious lack of low-hanging fruit. Ideas are more complicated and the simple ideas that a generalist has any chance to find have to a large extent already been discovered. Note also that in fact it is well before 100 years ago that this trend already started. Darwin, Maxwell, Faraday and many other 19th century researchers were already specialists by most notions of the term. So really this trend has been going on for almost 200 years.