LESSWRONG
LW

Comment Permalink
danohu20

As Zian says: large/liquid/free prediction markets aren't sufficient, they also need to be trusted by enough people, or by powerful enough people. IMO there will be at least a 7 year lag between prediction markets working well, and them being broadly accepted outside of narrow wonk/nerd circles.

In the existing world, covid was already front-page news in Jan/Feb 2020, with speculation about it spreading beyond China*. Few people in the West did anything. The limiting factor wasn't warnings being issued, it was people being able to grok that something Really Bad was coming, and prepare rather than burying heads in the sand.

Suppose the headlines said "prediction markets expect covid to reach us" rather than "experts expect covid to reach us". Who would have behaved differently? The kind of people and institutions which tend to react to signals from stock markets and opinion polls -- the competent minority.

Plus, even a good prediction market would not have immediately reached certainty that covid would become a pandemic. I imagine the alarm and confidence gradually increasing over early 2020

Overall, we might have shifted the reactions forward by a couple of weeks. In a really good scenario that might have led to enough pressure to make governments do something (e.g. widespread testing before it was too late). But...even that might have been enough to stop the pandemic in the spring.

 

(*) incidentally, I'm struggling to check my memory on that. Tips for how to not just confirm there was reporting on covid, but get a sense of how prominent it was?

Reply
ChristianKl20

In the existing world, covid was already front-page news in Jan/Feb 2020, with speculation about it spreading beyond China*. Few people in the West did anything. 

The stock market is essentially a prediction market so a seperate prediction market on the question whether COVID-19 reaches us wouldn't necessarily produced more information. 

Reply
1mike_hawke
I have to question the premises and the strength of the conclusion here. This comparison feels off--I remember things being much more controversial and confusing than "experts expect covid to reach us". But to answer the second part, I guess I would have to ask myself how many public health officials, supply chain influencers, lawmakers, etc. were in the competent minority (and how money-motivated they were). But certainty is never necessary for action. I intuitively imagine a major difference between the sort of uncertainty I felt in Feb 2020 (due to model uncertainty, low trust, competing heuristics, Twitter fog) vs. the uncertainty I would have if prediction markets had said "X% chance of pandemic spread within 3 months". As a human, these different types of uncertainty affect my behavior differently. I was able to replace some of the former uncertainty with the latter uncertainty by watching metaculus, but being able to freely hedge my bets would have been even better. Right, a 14-day shift can make a lot of difference in the early stages of exponential growth. (Or did you mean to say "not enough"?) I think there may be a decent case for prediction markets having only minor overall impact during the notional 7-year lag, but not a strong case. And even if prediction markets wouldn't have prevented any of the largest mistakes...it would be nice to get money for being correct. Or to be able to easily hedge between monetary risk (losing bets), health risks (virus), and professional risks (refusing to go to work in person). ...And to be honest it would have been real nice to ask Vox writers to put their money where their mouth was.
See in context
by mike_hawke
1 min read
7

10

In WebMD and the Tragedy of Legible Expertise, Scott Alexander ends by saying that we should be grateful for the experts we have, but that prediction markets would still be better.

Is he right? If so, how much better would prediction markets be? Let's do some counterfactual history. Conditional on large, liquid, free prediction markets being made suddenly available to the public in Nov 2019 (or April 2020, or any other date), what would have been the most likely pandemic outcomes? I'm interested in both vague and specific answers. Even better if you can estimate a quantity of money or life-years saved.

Covid-19
Personal Blog

10

How would free prediction markets have altered the pandemic?
New Comment
7 comments, sorted by
top scoring
Click to highlight new comments since:
[-]PeterMcCluskey220

Here are some pairs of contracts that would have been informative last spring:

  • Vaccinations in 2020 if human challenge trials are started by June 1.

  • Vaccinations in 2020 if human challenge trials are not started by June 1.

  • COVID deaths in 2020 if rapid at-home tests are approved by July 1.

  • COVID deaths in 2020 if rapid at-home tests are not approved by July 1.

  • COVID deaths in 2020 if most states keep schools closed all year.

  • COVID deaths in 2020 if most states reopen schools in September.

  • Vaccinations in winter 2021 if vaccine manufacturers are paid at least $1000 per dose for the first 10 million doses.

  • Vaccinations in winter 2021 if vaccine manufacturers are paid at most $100 per dose for the first 10 million doses.

Some contracts that would have been informative in October:

  • COVID deaths in 2021 if state health agencies are put in charge of most vaccinations.
  • COVID deaths in 2021 if drug stores are put in charge of most vaccinations.
  • COVID deaths in 2021 if hospitals are put in charge of most vaccinations.

It would likely require decades of advocacy by prediction market supporters before mainstream opinion shapers feel constrained to acknowledge that these markets constitute expert opinions. But if they did acknowledge that expertise enough for it to alter government policies, I'd guess that 2 or 3 of those policy changes would have cut COVID deaths by about 10% each.

Reply
[-]ChristianKl20

It would likely require decades of advocacy by prediction market supporters before mainstream opinion shapers feel constrained to acknowledge that these markets constitute expert opinions. 

I don't think it would take that long. If a prediction market says X is correct and a decision makers decides for Y they have afterwards a lot less excuses for making the wrong decision. It's easy for political opponents to write articles saying that the person was stupid for making the decision as the prediction market said something else. Articles that are easily written without doing any hard research. 

Blame minimization is very important for individual decision makers so it will affect decision making even when it won't always get people to follow the predicion market. 

Reply
[-]danohu20

As Zian says: large/liquid/free prediction markets aren't sufficient, they also need to be trusted by enough people, or by powerful enough people. IMO there will be at least a 7 year lag between prediction markets working well, and them being broadly accepted outside of narrow wonk/nerd circles.

In the existing world, covid was already front-page news in Jan/Feb 2020, with speculation about it spreading beyond China*. Few people in the West did anything. The limiting factor wasn't warnings being issued, it was people being able to grok that something Really Bad was coming, and prepare rather than burying heads in the sand.

Suppose the headlines said "prediction markets expect covid to reach us" rather than "experts expect covid to reach us". Who would have behaved differently? The kind of people and institutions which tend to react to signals from stock markets and opinion polls -- the competent minority.

Plus, even a good prediction market would not have immediately reached certainty that covid would become a pandemic. I imagine the alarm and confidence gradually increasing over early 2020

Overall, we might have shifted the reactions forward by a couple of weeks. In a really good scenario that might have led to enough pressure to make governments do something (e.g. widespread testing before it was too late). But...even that might have been enough to stop the pandemic in the spring.

 

(*) incidentally, I'm struggling to check my memory on that. Tips for how to not just confirm there was reporting on covid, but get a sense of how prominent it was?

Reply
[-]ChristianKl20

In the existing world, covid was already front-page news in Jan/Feb 2020, with speculation about it spreading beyond China*. Few people in the West did anything. 

The stock market is essentially a prediction market so a seperate prediction market on the question whether COVID-19 reaches us wouldn't necessarily produced more information. 

Reply
[-]mike_hawke10

I have to question the premises and the strength of the conclusion here.

Suppose the headlines said “prediction markets expect covid to reach us” rather than “experts expect covid to reach us”. Who would have behaved differently? The kind of people and institutions which tend to react to signals from stock markets and opinion polls—the competent minority.

This comparison feels off--I remember things being much more controversial and confusing than "experts expect covid to reach us". But to answer the second part, I guess I would have to ask myself how many public health officials, supply chain influencers, lawmakers, etc. were in the competent minority (and how money-motivated they were).

even a good prediction market would not have immediately reached certainty that covid would become a pandemic.

But certainty is never necessary for action. I intuitively imagine a major difference between the sort of uncertainty I felt in Feb 2020 (due to model uncertainty, low trust, competing heuristics, Twitter fog) vs. the uncertainty I would have if prediction markets had said "X% chance of pandemic spread within 3 months". As a human, these different types of uncertainty affect my behavior differently. I was able to replace some of the former uncertainty with the latter uncertainty by watching metaculus, but being able to freely hedge my bets would have been even better.

Overall, we might have shifted the reactions forward by a couple of weeks[...]even that might have been enough to stop the pandemic in the spring.

Right, a 14-day shift can make a lot of difference in the early stages of exponential growth. (Or did you mean to say "not enough"?)

I think there may be a decent case for prediction markets having only minor overall impact during the notional 7-year lag, but not a strong case.

And even if prediction markets wouldn't have prevented any of the largest mistakes...it would be nice to get money for being correct. Or to be able to easily hedge between monetary risk (losing bets), health risks (virus), and professional risks (refusing to go to work in person). ...And to be honest it would have been real nice to ask Vox writers to put their money where their mouth was.

Reply
[-]Zian20

large, liquid, free prediction markets being made suddenly available to the public

I am going to be pessimistic and guess that very little would change because this scenario does not include any change to the type of people who take prediction markets seriously and we already have some prediction markets.

Specifically, I predict that if there were an absolutely perfect prediction market for "Vaccine X will be X% effective" and the resulting prediction (in January of 2020) was mostly 98 to 99, the FDA would not behave any differently from what we observed with Pfizer or Moderna.

Reply
[-]ChristianKl20

Predictions about when how much vaccine doses are available given certain actions could have helped us having enough vaccination doses.

Reply
Moderation Log
Curated and popular this week
7Comments