you can always make predictions conditional on "no singularity"
This is true, but then why not state "conditional on no singularity" if they intended that? I somehow don't buy that that's what they meant
How much do you think subjective experience owes to the internal-state-analyzing machinery?
I'm actually not really sure. I find it plausible that subjective experience could exist without internal-state-analyzing machinery, and that's what I'm hypothesizing is going on with LLMs to some extent. I think they do have some self-model, but they don't seem to have access to internal states the way we do. Although I somehow think it's more likely that an LLM experiences something than a tree experiences something.
if I wanted to inspect that subjective experience, I probably couldn't do it by strapping a Broca's area etc.
I maybe agree with that, conditional on trees having subjective experience. What I do think might work is doing something more comprehensive: maybe bootstrapping trees with a lot more machinery that includes something to form concepts that correspond to whatever processes are leading to their experiences (insofar as there are processes corresponding to experiences. I'd guess things do work in this way, but I'm not sure). That machinery needs to be somehow causally entangled with those processes; consider how humans have complicated feedback loops such as touch-fire -> pain -> emotion -> self-modeling -> bodily-reaction -> feedback-from-reaction-back-to-brain...
The introspection, self-modeling, and language-production circuitry isn't an impartial window into what's going on inside, the story it builds reflects choices about how to interpret its inputs.
Yeah, that seems true too, but I guess if you have a window at all, then you still have some causal mechanism that goes from internal states corresponding to experiences to internal concepts correlated to those, which might be enough. Now, though, I'm pretty unsure whether the experience is actually due to the concepts themselves or the states that caused them, or whether this is just a confused way of seeing the problem.
I'm in an uncomfortable situation of having a better-argued and more convincing answer than any of the comments here, but I'm unable to trust it fully. ChatGPT o3 gave it. The prompt was simply my post, title included. Here is the answer:
1. What GDP actually measures
GDP is value added at market prices. Statistical agencies report it two ways:
- Nominal GDP – current‐dollar spending.
- Real GDP – nominal GDP deflated by a price index so that what is left is (roughly) the volume of goods and services, not their price.
Because the deflators strip out economy-wide price changes, a rise in real GDP cannot be caused by “just paying more for the same tomatoes.” In other words, inflation and real-GDP growth are separated by construction.
Where things get messy is measurement: digital goods are often free or priced in hard-to-track ways, so the official statistics probably under-state their contribution to welfare rather than over-state it. Experiments that impute the willingness-to-pay for free online services show real growth would have been 0.1-0.2 pp a year higher in the 2005-15 period if that value were included Bureau of Economic AnalysisNBER.
2. How a big metaverse sector would affect the real economy
- New income and purchasing power
If avatars buy my hand-crafted dragon skins for €20, I have €20 of extra income in the physical world. As long as the central bank keeps the general price level stable, that €20 really is additional purchasing power. - Relative-price shifts, not general inflation
Digital production is almost infinitely scalable, so its labour productivity will soar. In a competitive labour market that lifts wages everywhere, including in low-productivity, very physical sectors such as farming. Those sectors therefore become relatively more expensive—the classic Baumol cost-disease pattern VoxChicago Booth. Groceries, haircuts and housing might take a bigger slice of the average wallet, but that does not mean the economy has merely “bid up prices.” The extra spending power created in the metaverse simply shows up partly as:- higher nominal prices for resource-constrained goods, and
- a much larger menu of cheap or free digital experiences that didn’t exist before.
What happens to physical-goods consumption?
- If supply is elastic (manufactured items, many foods) real quantities bought will still rise.
- If supply is inelastic (urban land, trophy wine) real quantities bought may be flat and prices will climb.
Either way, people are richer in utility terms because they now enjoy both their old physical bundle and a vast new bundle of virtual goods whose marginal resource cost is near zero.
3. Why real well-being can grow while resource use decouples
Economic growth has already been shifting from material to intangible output for decades—software, streaming, consulting, design. The metaverse would push that frontier further. Because these activities consume little energy, metal or land, GDP can rise without a proportional rise in resource throughput—exactly Noah Smith’s point. The OECD, IMF and others note that the challenge is not physical scarcity but statistical visibility of intangibles and data OECDIMF eLibrary.
4. Putting it together
- Real GDP per capita would rise if the metaverse creates goods and services that people voluntarily pay for.
- Physical-goods prices would probably rise relative to digital goods, but that is a Baumol-style shift, not runaway inflation.
- Overall welfare rises because people get a larger choice set—most of it delivered almost for free once the first copy is produced.
So the answer is yes, we would genuinely be richer, even if the number of apples harvested per hectare barely changes. The gains show up partly as higher real incomes that can (within supply limits) buy more physical goods, and partly—as economists increasingly argue—as consumer surplus from wholly new digital experiences that standard GDP still struggles to count.
I feel like you're trying to make a point, but I can't see it
I would be very interested in reading a much more detailed account of the events, with screenshots, if you ever get around to it
I'm imagining driving down to Mountain View and a town once filled with people who had "made it" and seeing a ghost town
I'm guessing that people who "made it" have a bunch of capital that they can use to purchase AI labor under the scenario you outline (i.e., someone gets superintelligence to do what they want).
But I can't help but feeling such a situation is fundamentally unstable. If the government's desires become disconnected from those of the people at any point, by what mechanism can balance be restored?
I'm not sure I'm getting the worry here. Is it that the government (or whoever directs superintelligences) is going to kill the rest because of the same reasons we worry about misaligned superintelligences or that they're going to enrich themselves while the rest starves (but otherwise not consuming all useful resources)? If that's this second scenario you're worrying about, that seems unlikely to me because even as a few parties hit the jackpot, the rest can still deploy the remaining capital they have. Even if they didn't have any capital to purchase AI labor, they would still organize amongst themselves to produce useful things that they need, and they would form a different market until they also get to superintelligence, and in that world, it should happen pretty quickly.
Naively extrapolating this trend gets you to 50% reliability of 256-hour tasks in 4 years, which is a lot but not years-long reliability (like humans). So, I must be missing something. Is it that you expect most remote jobs not to require more autonomy than that?
I tried hedging against this the first time, though maybe that was in a too-inflammatory manner. The second time
Sorry for not replying in more detail, but in the meantime it'd be quite interesting to know whether the authors of these posts confirm that at least some parts of them are copy-pasted from LLM output. I don't want to call them out (and I wouldn't have much against it), but I feel like knowing it would be pretty important for this discussion. @Alexander Gietelink Oldenziel, @Nicholas Andresen you've written the posts linked in the quote. What do you say?
(not sure whether the authors are going to get a notification with the tag, but I guess trying doesn't hurt)
Two decades don't seem like enough to generate the effect he's talking about. He might disagree though.