gwern, I find your position against bubbles to be incredibly unlikely, and that is post my studying economics and finance informally for the last 3 decades.
(Forgive me when I read this mentally as "And that is post my being a random Internet pundit for decades".)
As far as real estate bubble, first I would point at Mortgage Backed Securities (MBS) rather than the direct real estate market. These were rated AAA, insured for less than a penny on the dollar, and purchased by ancient and venerable banks and others. And then in 2007/2008 they almost uniformly as a class blew up. Returned pennies on the dollar. Caused multiple firms and banks around the world to go bankrupt.
I don't think it's very useful to define a 'bubble' as "any large price increase followed by a price decrease".
I'd rather use a more powerful EMH-focused definition: a bubble is large price increase which represents an inefficiency in the market which is predictable in advance (not in hindsight), exploitable, and worth exploiting. Merely pointing out some disaster, or some large price decrease, does not demonstrate the existence of bubbles, because that observation could result from unavoidable or unobjectionable causes like the inherent consequences of risk-taking, mistaken analyses, perverse incentives, etc.
People make mistakes; disasters happen. If they never happened, and AAA never went bust, couldn't one make a lot of money by exploiting that inefficiency in the market and picking up pennies in front of the non-existent steamroller?
I am thinking of Buffett and Munger referring to MBS derivatives as Weapons of Financial Mass Destruction BEFORE the blowup, and I had in print in a book printed before the destruction a speecy by Munger talking about how there was going to be a tremendously horrible event because of derivatives "in the next 5 to 10 years" in a speech he gave in I think 2002. While MBS were hot, they were so in demand that brokers such as Salomon would create "synthetic" MBS, which were essentially just well documented bets that would pay off exactly as an MBS would pay off over their life, but were made up because there was still demand for MBS even after the last homeless person with a pulse in the US had been given a 100% non-doc mortgage to buy a house which would not be sellable for even 80% of what was financed two years later.
How much money did Munger & Buffet make off their shorts of housing, exactly? How much has Paulson made post-housing? (Does making billions off housing, and then losing billions on gold & China, look more like skill & inefficient markets or luck & selection effects?) How many economists did one hear of post-2008 who suddenly turned out to be Cassandras? You can go onto Bitcoin forums and tech websites right now, and watch people predict 20 out of the last 3 Bitcoin 'bubbles'. Finance is just the same. Post hoc selection of people warning something vaguely similar (derivatives? that's a rather roundabout way of predicting a housing bubble, which could have been powered by all sorts of financial instruments, not just derivatives) is worthless.
Is even this not a bubble? Not the market chasing a dream instead of a business proposition and trying to fly up to heaven with the dream and failing?
Housing prices in SF, Australia, London, Canada, Manhattan, China are holding steady at bubblelicious prices or trying to fly up to heaven. (Again, I borrow this point from Sumner.) Perhaps they are using technology from the Apollo program.
The NASDAQ composite peaked in early 2000 at over 4000. More than 13 years later it is STILL not back up to that level. Perhaps at least some of the investors in AMZN and AAPL in 1999 were not caught in a bubble, but what about the bulk of the money, of which about 70% of the value evaporated in less than 3 years, and which on the whole has not crept back up to even yet?
Why is this not just mistaken beliefs about the value of those loser companies and about high-tech business models? (Notice how the big IPOs lately all have pretty clear revenue streams from advertising.) How could one know in advance that Pets.com would not be Amazon.com, or vice-versa? How does a VC know which of his investments will go bankrupt and which will own an industry? Tell me: if tomorrow a break is discovered in the core Bitcoin protocol/cryptography and the price goes to $0.00, was Bitcoin a bubble or a mistake?
To summarize: I think you are grasping at surface features, not thinking about the anti-bubble arguments or are just unfamiliar, and are engaged in post hoc analysis where you select out of the buzzing hive of argument and disagreement a few strands which seem right to you with the benefit of many years of data.
OK you like EMH so much that you think 9 students from one professor all outperforming for decades is cherry picking and data mining. I think it is finding a small group of people wh oclaim to be learning from someone who has empirically verified methods, and who, when they apply these methods, get the predicted results consistently for decades. I think characterizing this as cherry picking and data mining is at more likely to be a bad explanation for what is being seen than is mine, which is that they are doing what ehy say they are doing, and it is wo...
In an unrelated thread, one thing led to another and we got onto the subject of overpopulation and carrying capacity. I think this topic needs a post of its own.
TLDR mathy version:
let f(m,t) be the population that can be supported using the fraction of Earth's theoretical resource limit m we can exploit at technology level t
let t = k(x) be the technology level at year x
let p(x) be population at year x
What conditions must constant m and functions f(m,k(x)), k(x), and p(x) satisfy in order to insure that p(x) - f(m,t) > 0 for all x > today()? What empirical data are relevant to estimating the probability that these conditions are all satisfied?
Long version:
Here I would like to explore the evidence for and against the possibility that the following assertions are true:
Please note: I'm not proposing that the above assertions must be true, only that they have a high enough probability of being correct that they should be taken as seriously as, for example, grey goo:
Predictions about the dangers of nanotech made in the 1980's shown no signs of coming true. Yet, there is no known logical or physical reason why they can't come true, so we don't ignore it. We calibrate how much effort should be put into mitigating the risks of nanotechnology by asking what observations should make us update the likelihood we assign to a grey-goo scenario. We approach mitigation strategies from an engineering mindset rather than a political one.
Shouldn't we hold ourselves to the same standard when discussing population growth and overshoot? Substitute in some other existential risks you take seriously. Which of them have an expectation2 of occuring before a Malthusian Crunch? Which of them have an expectation of occuring after?
Footnotes:
1: By carrying capacity, I mean finite resources such as easily extractable ores, water, air, EM spectrum, and land area. Certain very slowly replenishing resources such as fossil fuels and biodiversity also behave like finite resources on a human timescale. I also include non-finite resources that expand or replenish at a finite rate such as useful plants and animals, potable water, arable land, and breathable air. Technology expands carrying capacity by allowing us to exploit all resource more efficiently (paperless offices, telecommuting, fuel efficiency), open up reserves that were previously not economically feasible to exploit (shale oil, methane clathrates, high-rise buildings, seasteading), and accelerate the renewal of non-finite resources (agriculture, land reclamation projects, toxic waste remediation, desalinization plants).
2: This is a hard question. I'm not asking which catastrophe is the mostly likely to happen ever while holding everything else constant (the possible ones will be tied for 1 and the impossible ones will be tied for 0). I'm asking you to mentally (or physically) draw a set of survival curves, one for each catastrophe, with the x-axis representing time and the y-axis representing fraction of Everett branches where that catastrophe has not yet occured. Now, which curves are the upper bound on the curve representing Malthusian Crunch, and which curves are the lower bound? This is how, in my opinioon (as an aging researcher and biostatistician for whatever that's worth) you think about hazard functions, including those for existential hazards. Keep in mind that some hazard functions change over time because they are conditioned on other events or because they are cyclic in nature. This means that the thing most likely to wipe us out in the next 50 years is not necessarily the same as the thing most likely to wipe us out in the 50 years after that. I don't have a formal answer for how to transform that into optimal allocation of resources between mitigation efforts but that would be the next step.