Most, if not all of the nootropics out there are so-called "small molecule" drugs; their relatively simple molecular structure lends itself to easy synthesis (but wide-ranging, unpredictable effects within the human body).
Pharmaceuticals (and hopefully nootropics) in general are progressing toward more "large molecule" medicines (or "biologics" as this informative page on Bayer's site terms) that are much harder to design and manufacture, but have a more directed/precise effect within the human body.
All of the top "new" nootropics gwern listed (except phenylpiracetam) are peptides, which qualify as "large molecule"s. But, while we stumbled across many of those, in the future it should be feasible to design targeted peptides, taking into account the receptors we'd like to target, thus having fine-grained control over the effect of the drug.
The problem is that there are too many regulations and not nearly enough money in "nootropics" to attract the investment necessary develop a targeted nootropic peptide from the ground-up.
So Scott Alexander's post at http://slatestarcodex.com/2016/03/01/2016-nootropics-survey-results/ shows that the most "effective" "nootropics" have still been the ones that have existed for a long time. What do these results really mean, though? Is it possible that people are just worse at noticing the subtler effects of the other drugs, or are just much worse at disciplining themselves enough to correctly use the racetams or noopept (as in, with choline)?
How much potential is there in innovation in nootropics? What is holding this innovation back, if anything? It feels like there hasn't been any real progress over the last 15 years (other than massively increased awareness), but could targeted drug discovery (along with people willing to be super-liberal with their experimentation) finally lead to some real breakthroughs?