Omitting death seems like a big deal. Very crudely, it looks like p=10^-4. It's said that society values each life at $5M, so that's E=-$500 already, but each individual likely values their own life a bit higher.
having an insurance policy increases the expected value of receiving a flu shot, as many insurance companies will completely cover the cost of receiving a flu shot.
That, and a little pondering why, is all you ever needed to know.
Actually estimating the utility of a vaccine is very difficult for individuals who are not complete shut-ins interacting with nobody (but then those people won't get sick in the first place), or individuals who aren't complete psychopaths without a job or with a job unusually resistant to damage from absence of coworkers (because even a psychopath would generally not want to get the co-workers sick). This is because a vaccine massively affects the ability of the virus to spread to other people, and in the beginning phase of the epidemic one person infects more than other person. It matters a great deal to an insurance company (or a government), so they can invest a lot of man-hours into modelling of the spread of the virus.
It is fortunate that the cost of the vaccine is so low and the illness is so common you arrive at the correct decision regardless.
the costs of providing palliative care to oneself, estimated at 100 USD
I don't think I've ever had flu --- certainly I have never been diagnosed with it --- but when I've had an incapacitating heavy cold, palliative care consists of staying in bed and surviving on half a can of soup a day, and maybe some over-the-counter symptomatic treatment. This agrees with the recommendations for actual flu here and produces a saving of money, not a cost.
In case it was not obvious, the correct takeaway from this article is that you should go and get a flu shot, if you haven't gotten one already this year. If you have already gotten a flu shot this year, and you reply to this comment with a message that states that you have done so, I would be more than happy to upvote you.
I got one and took my young son to get one as well. Convincing a child of the benefit of getting a shot or nasal spray makes for an excellent lesson in rationality since you must convince the kid to accept immediate and certain pain in return for a moderately reduced chance of experiencing a larger amount of future pain.
You don't use rational argument because it's the most effective way to convince a child, you use rational argument because it's the most effective way to teach a child the use of rational argument. (which as a side benefit, eventually makes rational argument the most effective way to convince the child)
Judging by some of the attitudes I have heard from my friends about getting a flu jab (similar to Lumifer's comment), I have found that it is actually more effective to encourage people to get flu jabs for the benefit to others via herd immunity, rather than by emphasising the benefit to them.
I don't know if this is maybe some kind of ego-bias in that healthy people underestimate their chances of getting sick, or that doing something for you doesn't get you fuzzies, whereas doing it to help sick people and babies does.
Of course I am in a country that won't...
I think the calculation somewhat under values the cost of being sick. If it's been a while since the last turn you were serriously sick it's easy to forget just how deeply unpleasant it is. I was really sick with food poisoning recently and would have happily paid >500 USD to avoid that. It's been a while since I had the flu, but I think putting the misery and suffering at less than 300 USD is not optimal modeling.
Huh, I'll look into this. At 33yo it never really occurred to me healthy people got flu shots. I have never had a shot or the flu, and the only people I know who have had other contributing weaknesses. I'm not saying I will necessarily get one, but this definitely opens the question for me. Thanks!
That was a very nice article, well-researched and thorough; However, I do think I need to point out that you have made an error in your maths. (Interestingly, the effect of the error was to increase the cost of getting a flu shot).
Specifically, take a look at figure one, at the probabilities. You have stated, earlier, that the flu injection has an efficacy of 60%; that is to say, 60% of people who get the flu shot, who would otherwise have got the flu, don't get the flu.
Given that 90% of people don't get the flu, whether they have the shot or not, this mea...
Getting complications (allergy, ...) from the flu shot should be included somewhere in the graph - it can't be totally screened away, and while it's unlikely to happen, it's not much more unlikely than dying from the flu.
But most importantly (and this time, arguing for the shot), the "herd immunity" effects should be considered. Even if just half of the people take the shot, those who because of the shot didn't get the disease won't be carriers and won't spread the disease to even those who did get the shot.
I'm not seeing where the probabilities in the decision tree come from. Does 10% of the unvaccinated population catch flu every year, and 6% of the vaccinated? Those seem extraordinarily high figures, and I don't see them in the text. The two figures together imply an efficacy of 40%, not the 60% cited. The connection of the other probabilities to the data is also not clear.
...Although the costs of the possible outcomes shown in Figure 1 were calculated under the assumption that the individual receiving the flu shout was uninsured, having an insurance policy
I have already gotten a flu shot this year, primarily because the cost of getting one is approximately 10 minutes and 0 USD (They're covered by cost of attendance at my university and in a very convenient location for me).
> The cost of catching the flu despite not receiving a flu shot was estimated as being 1000 USD for a healthy adult, given the cost of losing 4-5 days of productivity if one's time is valued at 20 USD/hour, plus the costs of providing palliative care to oneself, estimated at 100 USD, and the cost equivalent to the negative monetary value of experiencing a lower quality of life than usual while sick, estimated at 200 USD. <<
This is this bit that stands out the most to me as warranting some analysis. Maybe in the US things are very different, bu...
I think there's a pretty big mistake here - the value of not getting flu is a lot more than $200.
At a $5M value of life, each day is worth about $200, so 7 days of almost complete incapacitation is -$1400.
I would certainly pay $1400 upfront to make a bad flu just instantly stop.
What about the cost of pain from the flu shot? Based on my past experiences (all from childhood, so maybe not that accurate for me now), I would be willing to pay $20-$50 to avoid the pain from a shot. I also didn't find the flu that unpleasant, so I might only be willing to pay $120-150 to avoid it assuming no risk of death. It seems like the expected value of a flu shot is small enough for these sorts of subjective preferences to tip the balance in many cases.
Figure 1. Decision Tree for Assessing the Impact of Immunization in Healthy Adults
I'd like to know how elastic the result is for changing cost values. Did you plug in different values and what were the results?
Am I being particularly dense or are there no numbers in the form of "expected value of not getting the shot: $Y", "expected value of getting the short: $X" and "difference $(X-Y), therefore ...."
The costs in your calculations, they are costs for whom?
To make an obvious observation, medical insurance exists for a reason and a lot of people are employed in such a way that missing a week of work does not cost them $1000. Another lot of people isn't employed at all.
None of the meta-reviews examined in this report found any evidence that receiving an influenza vaccine can cause serious adverse responses in patients
What about the non-serious responses? How much would you pay to avoid a mild fever/etc, how often does it happen?
I haven't seen all the comments on this post nor kept up with the different versions, but: have you considered doing an analysis with added information as how long into the flu season the person is? That is, the likelihood of my getting the flu this year goes down as time goes on, while the costs of getting the shot remain constant: at which point would the costs outweigh the gains?
people who may experience life-threating allergic reactions to components of the flu shot [...]
None of the meta-reviews examined in this report found any evidence that receiving an influenza vaccine can cause serious adverse responses in patients
If it's possible to have a life-threatening allergic reaction to the flu shot, and that doesn't count as "serious adverse responses", what would have counted?
A healthcare professional will be able to tell you whether or not it is safe for you to receive a flu shot prior to you receiving the immunization.
Is this actually true? And you mean this is a routine part of the procedure? How is that done?
As of 11:30CST, 11/11/14, this cost-benefit analysis has been revised, in order to address concerns raised in the comments. See http://lesswrong.com/r/discussion/lw/l8k/expansion_on_a_previous_costbenefit_analysis_of/ for more on how the cost-benefit analysis was carried out, and on how varying certain parameters affected the determined expected value of receiving a flu shot.
Overview
The purpose of this post is to provide readers of LessWrong with a summary of what the literature has to say about the efficacy and safety of influenza vaccinations, as well as to weigh the costs of receiving yearly flu vaccinations against the benefits which healthy adults gain from vaccination. As illustrated in the "Cost-Benefit Analyses" section of this report, the expected value of receiving flu vaccinations is positive for healthy adults. Therefore, a further motivation for authoring this post is that writing this post may encourage LessWrong readers who have not yet been vaccinated this flu season to receive immediate vaccination.
Introduction and Review of Literature
Several meta-analyses on the efficacy and safety of live-attenuated influenza vaccines, trivalent inactivated influenza vaccines, and tetravalent inactivated influenza vaccines have been published within the last two years (see Coleman et. al, Demicheli et. al, Osterholm et. al). These meta- analyses reached broadly similar conclusions regarding the efficacy of flu vaccines, which groups were most at risk for being infected with influenza, the safety of being vaccinated, and the magnitude of social harm caused yearly by influenza. However, there was disagreement between some articles regarding whether or not vaccination of healthy adults against influenza should be pursued as a public health policy. Specifically, the Demicheli paper (wrongly) found "no evidence for the utilization of vaccination against influenza in healthy adults as a routine public health measure". The issue of whether or not healthy adults should receive flu shots will be examined in the "Cost-Benefit Analyses" section of this report.
While the severity of flu seasons varies greatly year-to-year, an average of 24,000 deaths from the flu occur yearly in the US (NCIRD); approximately 90% of these deaths are in people of at least 65 years of age (NCIRD, CDC Key Facts). For all flu seasons between 1976 and 2007, an average of 2,385 adults of ages 19-64 died each year from flu and flu-related causes in the US (Thompson). Between 5 and 20 percent of the US population becomes infected with flu virus each flu season (CDC Q&A).
The *efficacy* of a vaccine is a measure of how effective a vaccine is; if half of a population of 2,000,000 people were given a vaccine with 60% efficacy, and 100,000 of the 1,000,000 total unvaccinated people got sick, then 40,000 of the 1,000,000 vaccinated people would get sick, as well. Many sources report the average efficacy of the flu vaccine throughout the US population to be 60% (Demicheli et. al) or 59% (Osterholm et. al, Coleman et. al). The CDC reports that the flu vaccine is more efficacious in young adults (70-90% efficacy, depending on how closely active viruses match the ones included in the vaccines manufactured during a given season), and less efficacious in those over 65 years of age (NCIRD). This has led to increased efforts at targeting healthcare workers, nursing home attendants, and others who are in frequent contact with elderly persons for yearly vaccination.
While some health agencies only recommend that elderly, infants, healthcare workers, pregnant women, and adults with certain medical complications, such as respiratory diseases, receive flu shots, the CDC recommends that all people 6 months and older get a flu shot every year (CDC Key Facts). The value which such at-risk individuals gain from being immunized against the flu is higher than the value which healthy adults gain from receiving flu shots. Certain individuals with extremely rare conditions, such as Guillain-Barré Syndrome (GBS), or people who may experience life-threating allergic reactions to components of the flu shot, should not receive flu shot. A healthcare professional will be able to tell you whether or not it is safe for you to receive a flu shot prior to you receiving the immunization.
None of the meta-reviews examined in this report found any evidence that receiving an influenza vaccine can cause serious adverse responses in patients (Coleman et. al, Osterholm et. al, Demicheli et. al). Receiving the influenza vaccine is safe, and it is not at all possible to catch the flu from receiving an influenza vaccine (CDC). Flu shots can cause arm pain or soreness, and can cause headache, mild fever, and muscle pain (Coleman et. al, Demicheli et. al).
Cost-Benefit Analyses
Estimates of the expected monetary values of possible flu-related outcomes were calculated relative to the value of not getting sick despite not receiving a flu shot, which was defined as having a utility of 0 USD. All payoffs shown are the payoffs which an average individual would derive from experiencing particular outcomes, rather than the value which either society, employers, or other parties would gain from a given individual either getting sick or not. Probabilities were assigned to each outcome, as shown in Figure 1, and a calculation of the expected value of receiving or not receiving a flu shot in a given year was carried out. The motivation for simplifying the calculation of the expected value of receiving a flu shot by restricting the outcome space as shown in Figure 1 was to demonstrate that, despite using conservative estimates and ignoring certain benefits of vaccination in the model, the expected value of vaccination is still positive for healthy adults. Since other demographics are expected to benefit even more from receiving flu vaccinations than healthy adults benefit from receiving flu vaccinations, the fact that healthy adults would benefit from receiving yearly flu vaccinations strongly suggests that all individuals above 6 months of age would benefit from receiving flu shots, excepting e.g. patients with GBS or allergies to components of the flu shot.
The cost of getting a flu shot was calculated as being 30 USD, given that it costs around 20 USD to receive a flu shot out of pocket, and given that it takes around 30 minutes to get a flu shot at a clinic. I have estimated the value of one's time as being 20 USD/hour for this calculation.
The value of not feeling sick for 3-10 days was subjectively estimated as being 200 USD for those who caught the flu, yet did not receive a flu shot. The outcome in which one catches the flu despite receiving a flu shot was given a payoff of - 230 USD, which was calculated by adding the cost of being vaccinated against the flu to the cost of feeling sick from getting the flu, calculated above.
The value a given individual would gain from not dying was estimated as being 5,000,000 USD.
Figure 1. Decision Tree for Assessing the Impact of Immunization in Healthy Adults
Although the costs of the possible outcomes shown in Figure 1 were calculated under the assumption that the individual receiving the flu shout was uninsured, having an insurance policy increases the expected value of receiving a flu shot, as many insurance companies will completely cover the cost of receiving a flu shot. Some governmental health insurance programs do not cover the cost of flu vaccinations. If one has insurance which covers the cost of the flu vaccine, the expected value of being vaccinated against the flu rises by 20 USD.
There are several positive benefits of receiving flu shots which have not been included in the above model. In particular, being vaccinated against the flu protects others in your community from becoming sick; this effect is known as the herd immunity effect. Also, the above analysis assumed that an individual would not lose income from missing work due to being sick from the flu; the effect which making this assumption had on the cost-benefit analysis presented here is examined in the link given at in the first paragraph of this post. Lastly, receiving the flu vaccine provides one with a small degree of protection against influenza-like infections (Coleman et. al, Demicheli et. al); this positive effect of the flu vaccine was not considered in the above assessment of the costs and benefits associated with healthy adults receiving the flu vaccine.
Again, the above analysis of the expected utility of receiving the flu vaccine each year was conducted with conservative estimates and a simple model which did not take into account all of the benefits of receiving the flu vaccine; this was done to show that the expected gain from receiving a flu shot is positive in the general case, given uncharitable assumptions.
Author's Reflections
I only read the "Methods", "Findings", and "Interpretation" sections of the Lancet article, as I did not have access to the full text of this paper.
Before writing this article and conducting the research which necessarily had to be conducted before writing it, I would have estimated the prior probability of elderly people, infants, pregnant woman, and asthmatics receiving a net benefit from vaccination as being very high, and the prior probability of healthy adults receiving a net benefit from influenza vaccination as moderately high.
I was raised in a family which, in general, valued being healthy, and, in particular, valued the practice of keeping up to date on one's vaccinations. However, I do not believe that the conclusions of this report would have been different if I had not come from such a culture.
Further Considerations
While this report is complete, I could have been more thorough. Part of why I am publishing this post now, rather than conducting more research before doing so, is that I expect that conducting additional research would be very unlikely to cause me to change any of the major conclusions of this report. To say the same thing from a decision-theoretic standpoint, information which has a very low chance of making one change their mind about something has little value, and I think that reading more papers on this topic would have a very low chance of changing any of my opinions on this topic.
References
1. Centers for Disease Control and Prevention. Key Facts About Seasonal Flu Vaccine. http://www.cdc.gov/flu/protect/keyfacts.htm (accessed 11/9, 2014).
2. Centers for Disease Control and Prevention. Seasonal Influenza Q&A. http://www.cdc.gov/flu/about/qa/disease.htm (accessed 11/9, 2014).
3. Coleman, B.; Cochrane, L.; Colas, L. Literature Review on Quadrivalent Influenza Vaccines. Public Health Agency of Canada 2014.
4. Demicheli, V.; Jefferson, T.; Al-Ansary, L.; Ferroni, E. Vaccines for preventing influenza in healthy adults. Cochrane Library 2014.
5. Milenkovic, M.; Russo, A.; Elixhauser, A. Hospital Stays for Influenza, 2004. Agency for Healthcare Research and Quality 2006.
6. National Center for Immunization and Respiratory Diseases. Epidemiology and Prevention of Vaccine-Preventable Diseases. http://www.cdc.gov/vaccines/pubs/pinkbook/flu.html (accessed 11/9, 2014).
7. Osterholm, M. T.; Kelley, N. S.; Sommer, A.; Belongia, E. A. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. The Lancet infectious diseases 2012, 12, 36-44.
8. Thompson, M.; Shay, D.; Zhou, H.; Bridges, C. Estimates of Deaths Associated with Seasonal Influenza --- United States, 1976--2007. 2010.