Some possibilities:

• you're under-estimating how many top nootropics are new. Semax, Selank, Cerebrolysin, PRL-5-83, and phenylpiracetam were all developed fairly recently IIRC
• you have too stringent definitions of 'new'; given that nootropics can be invented anytime between 'today' and 10,000 BC (earliest known human drug use, alcohol), and that improved cognition or stimulation can be noticed much easier than a 1% reduction in mortality from something-something carcinoma (first described in 1931), the base-rate of discoveries per year should be low
• you're ignoring the role of diffusion: many of the noots have only recently become available outside a lab or Russian setting (see Yvain's "An Iron Curtain Has Descended Upon Psychopharmacology"), in part because of the chicken-and-egg thing - noot users aren't as enthusiastic as the research chemical communities in using themselves as human guinea pigs because without anyone having used it, there's little even presumptive safety information (along the lines of 'it's been sold for a while and no one's reported dying of it'), and without anyone using it, there continues to be a lack of any such safety info. So there will be long lag times, and any good nootropic invented recently will not be well-known. If a Russian lab invents something as good as modafinil tomorrow, how many years will it be before it shows up in the West at less than $10/dose and with a few thousand people intrepid enough to try it such that a few score of them will praise it on a survey? Consider modafinil, which you agree is effective; it was invented in the '80sish, approved in the USA ~'97, but only in 2005 do I recall seeing it starting to get uptake in the nootropic community, and it took until 2010 until you could call it reasonably mainstream & commonly used.

• this is an outgrowth of the general slowdown in pharmaceutical development (Eroom's law); to continue the Yvain theme, consider his "Prescriptions, Paradoxes, and Perversities" where he scrapes a bunch of psychiatric drug average ratings and finds that the best psychiatric drugs came out in the 1960s and ratings have dropped steadily. Whatever trend has caused the pharmaceutical industry to spend ever more on R&D while getting ever fewer great drugs out at the end is, perhaps directly, also causing nootropics.

  There are a lot of suggested causes. The most common one is the low-hanging fruit/better-than-the-Beatles one: all the good drugs have been found already. This seems implausible for nootropics: all the good nootropics were found at the same time as all the good heart attack / cancer / diabetes etc drugs...? One interesting one came up recently: https://www.reddit.com/r/DecisionTheory/comments/46ghnw/when_quality_beats_quantity_decision_theory_drug/ It's caused by not all the low-hanging fruit being exhausted and there simply not being many good drugs left to find, but by reductions in our ability to find the good drugs because of deceptively-small decreases in the predictive validity of the early screening stages of drug development resulting in good drugs being passed over regardless of how many chemicals get thrown into the initial screens.

  (This may also explain low rates of nootropics finds. If you find a fantastic sedative/anxiolytic, or a fantastic stimulant, you have a decent chance of noticing this in your standard rat/mice model - they run a lot more or less on their wheel. This is, IIRC, how modafinil was found: the mice moved around a lot more on it. But what sort of easy quick screen do you do for 'increases WM' or 'increases long-term memory retention' or 'increases IQ'? Yes, there are tests you can do for some of them in animal models, but they tend to be expensive and slow and noisy, like the Morris water test. You're not going to screen a hundred compounds a day with that, that's for sure.)

• nootropics are easy to find, but the Western medical system is based on bio-puritanism: you're only allowed to find treatments for official diseases.

  You aren't permitted to look for stuff which makes healthy people 'better than well'. You can't get Adderall or modafinil approved for healthy people, you can only get it approved for 'ADHD' or 'narcolepsy'. The FDA will not give you a patent for finding a stimulant better than nicotine, the patent you need to make back the R&D & turn a profit. Consider the difficulty in getting the FDA to sign off on a simple trial of metformin in healthy people; it's difficult because it's trying to make them 'better than well'. This is why you keep reading about nootropics getting tested in Alzheimer's trials; because Alzheimer's is an official Disease, you are allowed to talk about improving Alzheimer's patients' cognition (because you are treating their symptoms). This is similar to genetic engineering: it's OK to talk about editing embryos to fix genetic diseases, and somewhat OK to talk about preventing retardation or bringing up to baseline, but anything past that and you might as well grow a little mustache & plotting how to invade Eastern Europe. This is unhelpful for nootropics because while the Alzheimers studies might establish basic safety, they won't establish efficacy in healthy adults because Alzheimers patients are deeply neurobiologically damaged and this may mask any improvements (it would be like testing creatine's effects on running ability in people with broken legs). This is why we instead have to settle for tiny little psychology experiments with n=20.

  This might explain the Russian thing; the Communists were OK with the idea of being better than well and were willing to pay for research into performance enhancement (you can read about some of the related material in Stambler's A History of Life-Extensionism), and so when their researchers went looking, they found.

Incidentally, I should point out that the RC community is fascinating for another reason: the sheer volume they output. When I first heard about RCs, my attitude was 'so what, their favorite drugs will be banned within the year and then it'll be no different from the regular illegal drug scenes' but of course, the point of RCs is that they are literally able to find or innovate and produce new psychedelics and stimulants and sedatives faster than the global authorities can hear of and ban them - which makes them a fascinating counterexample to Eroom's law and any slowdown of nootropics. 41 in 2010, 49 in 2011, 57 in 2012 (Drugs 2.0), 56 in 2013... Or consider Shulgin's famous career; how much was that he was a genius, and how much was that he had minimal competition and was willing to take his own creations rather than faff around with cells in a petry dish? Some have become quite popular in their own right rather than just as copycats and have survived banning.

How do they do it? Most of it seems to be a willingness to do phenotypic assays in the most predictive animal models around (ie humans), iterate constantly on variants of previous drugs to produce analogues, and zero regulation or enforcement of patents with decentralized R&D. While it's probably easier to find a decent psychedelic or sedative than a nootropic or stimulant, it can't be that much easier.

If it was easy to make a human brain work better by tweaking a few chemicals, evolution probably would have done it.

We lack structured ways to communicate knowledge about the effects of new nootropics that we have for the official drugs.

I think that there is several answers:

1. There are some interesting new stuff, like NGF drops, but only a few people have tested them and results are not shown in the survey. We just don't have enough user experiences and existing ones are more anecdotic. http://www.longecity.org/forum/topic/72272-ngf-spray/

2. Also we now have very large and easily available base of knowledge and vendors for any nootropic, which make search and personal testing much easy, and provide better personal results with existing nootropics. Combining, cycling and personal targeting of nootropics provide much better personal results from my experiences.

3. There are several promising compounds in testing phase, like different AMPAkines.

4. Also I think that there were some sampling problems with the survey as it puts racetams so low.