Some possibilities:
this is an outgrowth of the general slowdown in pharmaceutical development (Eroom's law); to continue the Yvain theme, consider his "Prescriptions, Paradoxes, and Perversities" where he scrapes a bunch of psychiatric drug average ratings and finds that the best psychiatric drugs came out in the 1960s and ratings have dropped steadily. Whatever trend has caused the pharmaceutical industry to spend ever more on R&D while getting ever fewer great drugs out at the end is, perhaps directly, also causing nootropics.
There are a lot of suggested causes. The most common one is the low-hanging fruit/better-than-the-Beatles one: all the good drugs have been found already. This seems implausible for nootropics: all the good nootropics were found at the same time as all the good heart attack / cancer / diabetes etc drugs...? One interesting one came up recently: https://www.reddit.com/r/DecisionTheory/comments/46ghnw/when_quality_beats_quantity_decision_theory_drug/ It's caused by not all the low-hanging fruit being exhausted and there simply not being many good drugs left to find, but by reductions in our ability to find the good drugs because of deceptively-small decreases in the predictive validity of the early screening stages of drug development resulting in good drugs being passed over regardless of how many chemicals get thrown into the initial screens.
(This may also explain low rates of nootropics finds. If you find a fantastic sedative/anxiolytic, or a fantastic stimulant, you have a decent chance of noticing this in your standard rat/mice model - they run a lot more or less on their wheel. This is, IIRC, how modafinil was found: the mice moved around a lot more on it. But what sort of easy quick screen do you do for 'increases WM' or 'increases long-term memory retention' or 'increases IQ'? Yes, there are tests you can do for some of them in animal models, but they tend to be expensive and slow and noisy, like the Morris water test. You're not going to screen a hundred compounds a day with that, that's for sure.)
nootropics are easy to find, but the Western medical system is based on bio-puritanism: you're only allowed to find treatments for official diseases.
You aren't permitted to look for stuff which makes healthy people 'better than well'. You can't get Adderall or modafinil approved for healthy people, you can only get it approved for 'ADHD' or 'narcolepsy'. The FDA will not give you a patent for finding a stimulant better than nicotine, the patent you need to make back the R&D & turn a profit. Consider the difficulty in getting the FDA to sign off on a simple trial of metformin in healthy people; it's difficult because it's trying to make them 'better than well'. This is why you keep reading about nootropics getting tested in Alzheimer's trials; because Alzheimer's is an official Disease, you are allowed to talk about improving Alzheimer's patients' cognition (because you are treating their symptoms). This is similar to genetic engineering: it's OK to talk about editing embryos to fix genetic diseases, and somewhat OK to talk about preventing retardation or bringing up to baseline, but anything past that and you might as well grow a little mustache & plotting how to invade Eastern Europe. This is unhelpful for nootropics because while the Alzheimers studies might establish basic safety, they won't establish efficacy in healthy adults because Alzheimers patients are deeply neurobiologically damaged and this may mask any improvements (it would be like testing creatine's effects on running ability in people with broken legs). This is why we instead have to settle for tiny little psychology experiments with n=20.
This might explain the Russian thing; the Communists were OK with the idea of being better than well and were willing to pay for research into performance enhancement (you can read about some of the related material in Stambler's A History of Life-Extensionism), and so when their researchers went looking, they found.
Incidentally, I should point out that the RC community is fascinating for another reason: the sheer volume they output. When I first heard about RCs, my attitude was 'so what, their favorite drugs will be banned within the year and then it'll be no different from the regular illegal drug scenes' but of course, the point of RCs is that they are literally able to find or innovate and produce new psychedelics and stimulants and sedatives faster than the global authorities can hear of and ban them - which makes them a fascinating counterexample to Eroom's law and any slowdown of nootropics. 41 in 2010, 49 in 2011, 57 in 2012 (Drugs 2.0), 56 in 2013... Or consider Shulgin's famous career; how much was that he was a genius, and how much was that he had minimal competition and was willing to take his own creations rather than faff around with cells in a petry dish? Some have become quite popular in their own right rather than just as copycats and have survived banning.
How do they do it? Most of it seems to be a willingness to do phenotypic assays in the most predictive animal models around (ie humans), iterate constantly on variants of previous drugs to produce analogues, and zero regulation or enforcement of patents with decentralized R&D. While it's probably easier to find a decent psychedelic or sedative than a nootropic or stimulant, it can't be that much easier.
the Communists were OK with the idea of being better than well and were willing to pay for research into performance enhancement (you can read about some of the related material in Stambler's A History of Life-Extensionism), and so when their researchers went looking, they found.
What's the attitude of present day China when it comes to performance enhancement? Shouldn't they also care about it?
I don't know too much about the East Asian countries other than culturally, they seem to be massively anti-drugs of any kind except tea & tobacco (with a minor exception for amphetamines because they help you work harder); every time I come across a discussion of drugs, especially psychedelics, in a Japanese or Chinese fiction or nonfiction work, I am struck by how ignorant the author obviously is and how they are peddling out of date War on Drugs & Reefer Madness hysteria.
In China, this might have something to do with the nationalist propaganda and heavy emphasis on the Opium Wars & foreign devils peddling evil foreign drugs and how the Community party Made China Great Again, but that wouldn't explain Japan or how the South Korean media goes into raptures when a Canadian is caught with 1 marijuana plant in his closet. The Chinese government also has an interesting conflict of interest: they don't necessarily want to invent or fund a better stimulant because that would compete with the source of something like 10% of all government revenue - the government tobacco monopoly.
they seem to be massively anti-drugs of any kind except tea & tobacco
Which is interesting as the Chinese traditional medicine heavily relies on hundreds of plant preparations, some with (claimed) nootropic properties.
Also, when drug use became socially acceptable in the West? 1960s, I guess, after Kerouac, Burroughs, etc?
Also, when drug use became socially acceptable in the West? 1960s, I guess, after Kerouac, Burroughs, etc?
Way before that. Drug use has always been a strand in the West: the Eleusianian mysteries, etc. Think William James on nitrous oxide, Freud and cocaine, heroin and opium in widespread use by regular people...
Yes, that's true, during late XIX - early XX century drugs like cocaine and opium were in (relatively) common use. But that raises the question whether most of the XX century was an aberration in that respect, a temporary victory of puritans...
Psychedelics are a bit of a special case though, in that they became known and widespread quite late. I am sure there were some Victorian gentlemen who tried magic mushrooms and such, but it took LSD (and cheap LSD) to get psychedelics to the masses. And that happened in the West -- I don't know if East Asia ever had a wave of people experimenting with psychedelics. China was too poor and Japan was too conformist.
The FDA will not give you a patent
The issue isn't patents (which are not awarded by the FDA anyway) but whether they will give your drug approval to market.
If it was easy to make a human brain work better by tweaking a few chemicals, evolution probably would have done it.
Humans are in a very different situation than they were for most of their evolutionary lifespan.
For one: there is an abundance of resources: tweaks that increase brainpower at the cost of calories which might kill on the Savannah would be solid gold by today's standards. You'd be smarter AND hotter!
For two: the kind of things we want people to use their brainpower on are not the same things the brain was evolved to optimize to think about. For example, despite being dis-tractable, my girlfriend can concentrate on and pay attention to animals for hours, and has a strong desire to walk miles and miles every day. I'm sure this was very handy for persistence hunting, but it's a handicap when it comes to working in a bank. And so Vyvanse comes to the rescue: not as a PURE nootropic but one that allows us to make different neural tradeoffs than evolution was interested in.
For three: Human intelligence already varies quite a bit, much of it achieved by 'tweaking' a few chemicals (DNA, for one). We don't understand the neurological difference between John Von Neumann and an average person, but I think there's a decent chance that within that variation is not just genetic/developmental differences but ones in neurochemistry that can be duplicated in others.
Finally: evolution is incapable of exploring chemical-space by more than a few compounds per generation. Despite their utility, humans never naturally evolved to produce penicillin, or caffeine, or opiates when needed. There is a HUGE population of chemicals that the human body has never tried producing which we can utilize. The argument that "If this was possible, evolution would have done it" proves far too much.
We know that the variation in any single locus is responsible for < 1/100 of the variance of IQ. If genes corresponded to drugs, then that gives an upper bound on the efficacy of drugs. I think that we can agree that 100 does not counts as "a few."
I believe CronoDAS is referring to Algernon's Law. Gwern describes the issues pretty well here, including several classes on "loopholes" we might employ to escape the general rule.
The classifications of different types of loopholes is still pretty high level, and I'd love to see some more concrete and actionable proposals. So, don't take this as saying "this is old hat", but only as a jumping off point for further discussion.
Probably because of different genes, which are thousands, and different early development wiring and education. It can't be replaced by a few drugs.
Miller's point being that those thousands of genes can easily be driven to fixation by evolution within a fairly short time, yet have not, and it's not clear from the GWASes yet if they're even under directional selection.
Right now, between the GCTAs and the failure to find lots of important rare variants affecting intelligence such as mutation load (eg no Swedish paternal age effect, unlike many disorders), the consensus seems to be swinging towards some sort of frequency-dependent or stabilizing selection: greater intelligence comes with some sort of fitness penalty (greater energetic consumption?) or maybe greater vulnerability to developmental disruption through poor environment and so a net disadvantage which favors poorer but more robust variants (and eventually, canalization). Given the accumulation of archaic & ancient genomes and further intelligence GWASes, we may be able to get a definitive answer to the old puzzle of why intelligence is heritable at all in the next few years.
Hence my old point about nootropics: for the effective ones, the reason the evolutionary argument fails may simply be that they require more metabolic resources which would be a fitness disadvantage but that no longer applies in the modern calorie-overload environment.
This isn't really an answer. There are nootropics which at least seem to make the human brain work better. The question is why new ones aren't being invented. Are the existing ones really the best possible?
And yes it's unlikely there are drugs that can magically increase IQ. But there are many other effects drugs can have other than improving IQ. Like stimulants seem to improve focus almost magically for some people.
Even the thing about evolution isn't necessarily true. Evolution is slow and random, and isn't optimizing for IQ directly, but also other things like energy usage.
Most, if not all of the nootropics out there are so-called "small molecule" drugs; their relatively simple molecular structure lends itself to easy synthesis (but wide-ranging, unpredictable effects within the human body).
Pharmaceuticals (and hopefully nootropics) in general are progressing toward more "large molecule" medicines (or "biologics" as this informative page on Bayer's site terms) that are much harder to design and manufacture, but have a more directed/precise effect within the human body.
All of the top "new" nootropics gwern listed (except phenylpiracetam) are peptides, which qualify as "large molecule"s. But, while we stumbled across many of those, in the future it should be feasible to design targeted peptides, taking into account the receptors we'd like to target, thus having fine-grained control over the effect of the drug.
The problem is that there are too many regulations and not nearly enough money in "nootropics" to attract the investment necessary develop a targeted nootropic peptide from the ground-up.
We lack structured ways to communicate knowledge about the effects of new nootropics that we have for the official drugs.
I think nootropics like dihexa and sunifiram are extremely dangerous to experiment with. I choose to stick with well-known ones that work like phenylpiracetam and armodafinil :) http://nootropicnation.com
I think that there is several answers:
There are some interesting new stuff, like NGF drops, but only a few people have tested them and results are not shown in the survey. We just don't have enough user experiences and existing ones are more anecdotic. http://www.longecity.org/forum/topic/72272-ngf-spray/
Also we now have very large and easily available base of knowledge and vendors for any nootropic, which make search and personal testing much easy, and provide better personal results with existing nootropics. Combining, cycling and personal targeting of nootropics provide much better personal results from my experiences.
There are several promising compounds in testing phase, like different AMPAkines.
Also I think that there were some sampling problems with the survey as it puts racetams so low.
There are several promising compounds in testing phase, like different AMPAkines.
Ah yes. The Ampakines are the nootropics of the future - and always will be. I've been hearing since what must be the early '00s about how amazing the ampakines are. But aside from the -racetams...
So Scott Alexander's post at http://slatestarcodex.com/2016/03/01/2016-nootropics-survey-results/ shows that the most "effective" "nootropics" have still been the ones that have existed for a long time. What do these results really mean, though? Is it possible that people are just worse at noticing the subtler effects of the other drugs, or are just much worse at disciplining themselves enough to correctly use the racetams or noopept (as in, with choline)?
How much potential is there in innovation in nootropics? What is holding this innovation back, if anything? It feels like there hasn't been any real progress over the last 15 years (other than massively increased awareness), but could targeted drug discovery (along with people willing to be super-liberal with their experimentation) finally lead to some real breakthroughs?