Thanks for the link assistance.
I agree that my mathematics example is insufficient to prove the general claim: "One will master only a small number of skills". I suppose a proper argument would require an in-depth study of people who solve hard problems.
I think the essential point of my claim is that there is high variance with respect to the subset of the population that can solve a given difficult problem. This seems to be true in most of the sciences and engineering to the best of my knowledge (though I know mathematics best). The theory I believe that explains why this variation occurs is that the subset of people which can solve a given problem use unconscious heuristics borne out of the hard work they put into previous problems over many years.
Admittedly, the problems I am thinking about are kind of like NP problems: it seems difficult to find a solution, but once a solution is found we can know it when we see it. There tends to be a large number of such problems that can be solved by only a small number of people. And the group of people that can solve them varies a lot from problem to problem.
There are also many hard problems for which it is hard to say what a good solution is (e.g. it seems difficult to evaluate different economic policies), or the "goodness" of a solution varies a lot with different value systems (e.g. abortion policy). It does seem that in these instances politicians claim they can give good answers to all the problems as do management consulting companies. Public intellectuals and pundits also seem to think they can give good answers to lots of questions as well. I suppose that if they are right then my claim is wrong. I argue that such individuals and organizations claim to be able to solve many problems but since its hard to verify the quality of the solutions we should take the claim with a grain of salt. We know that individuals who can solve lots of problems would have a lot of status so there is a clear incentive to claim to be able to solve problems that one cannot actually solve if verifying the solution is sufficiently costly.
I also think there is a good reason to think that even for those problems whose solutions are difficult to evaluate we should expect only a small number of people to actually give a good solution. The reason relates to a point made by Robin Hanson (and myself in another comment) which is that in solving a problem you should try to solve many at once. A good solution to a problem should give insight to many problems. Conversely, to understand and recognize a good solution to a given hard problem one should understand what it says about many other problems. The space of problems is too vast for any human being to know but a small portion, so I expect that people who are able to solve a given problem should only be those aware of many related problems and that most people will not be aware of the related problems. Given that in our civilization different people are exposed to different problems (no matter in which field they are employed) we should expect high variance of who can solve which hard problems.
I've collected some tips and tricks for answering hard questions, some of which may be original, and others I may have read somewhere and forgotten the source of. Please feel free to contribute more tips and tricks, or additional links to the sources or fuller explanations.
Don't stop at the first good answer. We know that human curiosity can be prematurely satiated. Sometimes we can quickly recognize a flaw in an answer that initially seemed good, but sometimes we can't, so we should keep looking for flaws and/or better answers.
Explore multiple approaches simultaneously. A hard question probably has multiple approaches that are roughly equally promising, otherwise it wouldn't be a hard question (well, unless it has no promising approaches). If there are several people attempting to answer it, they should explore different approaches. If you're trying to answer it alone, it makes sense to switch approaches (and look for new approaches) once a while.
Trust your intuitions, but don't waste too much time arguing for them. If several people are attempting to answer the same question and they have different intuitions about how best to approach it, it seems efficient for each to rely on his or her intuition to choose the approach to explore. It only makes sense to spend a lot of time arguing for your own intuition if you have some reason to believe that other people's intuitions are much worse than yours.
Go meta. Instead of attacking the question directly, ask "How should I answer a question like this?" It seems that when people are faced with a question, even one that has stumped great minds for ages, many just jump in and try to attack it with whatever intellectual tools they have at hand. For really hard questions, we may need to look for, or build, new tools.
Dissolve the question. Sometimes, the question is meaningless and asking it is just a cognitive error. If you can detect and correct the error then the question may just go away.
Sleep on it. I find that I tend to have a greater than average number of insights in the period of time just after I wake up and before I get out of bed. Our brains seem to continue to work while we're asleep, and it may help to prime it by reviewing the problem before going to sleep. (I think Eliezer wrote a post or comment to this effect, but I can't find it now.)
Be ready to recognize a good answer when you see it. The history of science shows that human knowledge does make progress, but sometimes only by an older generation dying off or retiring. It seems that we often can't recognize a good answer even when it's staring us in the face. I wish I knew more about what factors affect this ability, but one thing that might help is to avoid acquiring a high social status, or the mental state of having high social status. (See also, How To Actually Change Your Mind.)