The TLDR has multiple conclusions but this is my winner:

My conclusion -- springing to a great degree from how painful seeking clear predictions in 700 pages of words has been -- is that if anyone says "I have a great track record" without pointing to specific predictions that they made, you should probably ignore them, or maybe point out their lack of epistemic virtue if you have the energy to spare for doing that kind of criticism productively.

There is a skill in writing things that, when read later, are likely to be interpreted as predictions of things that happened between writing and reading. This is the skill of astrologers. There is another skill in accurately predict the future, and writing that down. This is the skill of forecasters.

The post, and the comments and reviews on this post, show this off. People disagree with the author and each other. If there is serious debate over how a prediction resolves after the resolution time has passed, it might have been astrology the whole time.

This point is also made in Beware boasting about non-existent forecasting track records from 2022. I think this post adds substantial evidence to that perspective on forecasting in general.

As a side note, the author notes both Hanson and Yudkowsky are hard to score. This is a generalized problem that afflicts many smart people and maybe also you. It definitely afflicts me. It's not about one person.

Your calculations look right for Shapley Values. I was calculating based on Ninety-Three's proposal (see here). So it's good that in your calculations the sum of parts equals the combined, that's what we'd expect for Shapley Values.

There’s also a question about cross-domain transferability of good takes.

Agreed. That isn't a difference between contributing "considerations" and "predictions" (using Habryka's reported distinction). There are people who contribute good analysis about geopolitics. Others contribute good analysis about ML innovations. Does that transfer to analysis about AGI / ASI? Time will tell - mostly when it's already too late. We will try anyway.

In terms of predicting the AI revolution an important consideration is what will happen to power. Will it be widely or narrowly distributed? How much will be retained by humans? More importantly, can we act in the world to change any of this? These are similar to geopolitical questions, so I welcome analysis and forecasts from people with a proven track record in geopolitics.

The industrial revolution is a good parallel. Nobody in 1760 (let alone 1400) predicted the detailed impacts of the industrial revolution. Some people predicted that population and economic growth would increase. Adam Smith had some insights into power shifts (Claude adds Benjamin Franklin, François Quesnay and James Steuart). That's about the best I expect to see for the AI revolution. It's not nothing.

I don't think we disagree on culture. I was specifically disagreeing with the claim that Metaculus doesn't have this problem "because it is not a market and there is no cost to make a prediction". Your point that culture can override or complement incentives is well made.

The cost to make a prediction is time. The incentive of making it look like "Metaculus thinks X" is still present. The incentive to predict correctly is attenuated to the extent that it's a long-shot conditional or a far future prediction. So Metaculus can still have the same class of problem.

If anyone's tracking impact, my family had five people tested due in large part due to this post, of whom five were low and started supplementing. We're not even vegan.

Would you, failing to observe anything on the subject after a couple of hours of Googling, conclude that your civilization must have some unknown good reason why not everyone was doing this already?

No, but not for "Civilizational Adequacy" reasons. In a hypothetical civilization that is Adequate in the relevant ways, but otherwise like ours, I would also not conclude that there was an unknown good reason why not everyone was doing this already. Here's a simple model to apply for many values of X:

• If civilization has an unknown-to-me reason why X is a good idea, I expect to observe search results saying that X is a good idea and giving the reason.
• If civilization has an unknown-to-me reason why X is a bad idea, I expect to observe search results saying that X is a bad idea and giving the reason.
• If civilization does not know if X is a good or bad idea, I expect to observe no search results, or mixed search results.

I don't see any way for me to conclude, from a lack of search results, that civilization must have some unknown good reason why not everyone was doing this already.

I tried to disprove this by thinking of a really random and stupid X. Something so randomly stupid that I could be the first person in civilization to think of it. The idea of "inject bleach to cure covid" was already taken. What if I generalize to injecting some other thing to cure some other thing? My brain generated "cure cancer by injecting hydrogen peroxide". No, sorry, not random and stupid enough, the internet contains Does Hydrogen Peroxide Therapy Work?, it was the first search result.

More randomness and stupidity needed! How about putting cats in the freezer to reduce their metabolism and therefore save money on cat food? Well, yes this appears to be a new innovation in the field of random stupid ideas. On the other hand the first ten search results included Cat survives 19-hour ordeal in freezer so civilization has a reason why that is a bad idea and it's available for anyone who searches for it.

I'm obviously not creative enough so I asked Claude. After a few failed attempts (emojis in variables names! jumping jacks before meetings!) we got to:

Improve remote work meetings by requiring everyone to keep one finger touching their nose at all times while speaking, with their video on. Missing the nose means you lose speaking privileges for 30 seconds.

Success: a truly novel, truly terrible idea. In this case, civilization has multiple good reasons why not everyone is doing this already, but there aren't any specific search results on this specific idea. Even then, if I spend a couple of hours searching and reading, I'm going to hit some tangential search results that will give me some hints.

I don't think this proposal satisfies Linearity (sorry, didn't see kave's reply before posting). Consider two days, two players.

Day 1:

• A => $200
• B => $0
• A + B => $400

Result: $400 to A, $0 to B.

Day 2:

• A => $100
• B => $100
• A + B => $200

Result: $100 to A, $100 to B.

Combined:

• A => $300
• B => $100
• A + B => $600
• So: Synergy(A+B) => $200

Result: $450 to A, $150 to B. Whereas if you add the results for day 1 and day 2, you get $500 to A, $100 to B.

This is excellent. Before reading this post in 2023, I had the confusion described. Roughly, that Aumann agreement is rationally correct, but this mostly doesn't happen, showing that mostly people aren't rational. After reading this post, I understood that Aumann agreement is extremely common, and the exceptions where it doesn't work are best understood as exceptions. Coming back to read it in 2024, it seems obvious. This is a symptom of the post doing its job in 2023.

This is part of a general pattern. When I think that human behavior is irrational, I know nothing. When I see how human behavior can be modeled as rational, I have learned something. Another example is how people play The Ultimatum Game. When I was shown how turning down an "unfair" share can be modeled as a rational response to coercion, I had a better model with better predictions and a better appreciation of my fellow humans.

The post is short, clearly written, seeks to establish a single thing, establishes it, and moves on without drama. Perhaps this is why it didn't get a lot of engagement when it was posted. The 2023 review is a chance to revisit this.

I could build on this post by describing how Aumann agreement occurs in prediction markets. On Manifold there are frequently markets where some group of people think "90% YES" and others think "90% NO" and there are big feelings. If this persists over a long enough period, with no new evidence coming in, the market settles at some small percentage range with people on both sides hiding behind walls of limit orders and scowling at each other. To some extent this is because both sides have built up whatever positions satisfy their risk tolerance. But a lot of it is the horrible feeling that the worst people in the world may be making great points.