Any thoughts on how to adjust this for recent findings that omicron is more susceptible to testing negative in the early stages of infectiousness?
The findings:
Excellent question!
My best guess: for detecting people who are infectious but asymptomatic, antigen tests will likely perform approximately as well with Omicron as they have with Delta. Because Omicron infections ramp up so fast, however, I'm reducing my guess for how long you can trust the results from 12 hours to 6 hours. (That is to say, if you tested negative this morning, you shouldn't assume that you aren't infectious this evening).
In addition to the data you cite, Abbott claims their testing shows no decrease in BinaxNOW effectiveness against Omicron. That would also be my prior. I'm curious what the FDA has found, although they've been coy about releasing details. I assume we'll see more data soon.
Update 2022-01-07: It's pretty clear now that antigen tests are less effective against Omicron, probably for three reasons: they are less sensitive to Omicron, viral loads rise faster with Omicron than with Delta, and Omicron is more infectious than Delta.
Based on the current data, I'm changing two things in the original writing, with the caveat that these are just guesses that are not backed up by hard data:
Personally, I've started using the throat swab technique: https://www.youtube.com/watch?v=5qHTBlxfNes That technique was likely more effective for detecting Delta, and there are strong theoretical reasons to think it might be much more effective for detecting Omicron.
Introduction
Home antigen tests for COVID are an imperfect but useful tool. In this post I’ll discuss the four scenarios where I think they’re most useful, share a few thoughts about using them correctly, and finish by taking a deep look at the data on accuracy.
If you don’t already understand concepts like sensitivity and positive predictive value, you might want to read this first.
I’ll focus on the Abbott BinaxNOW test because I think it’s overall the best and most available home antigen test in the US as of October 2021 (the situation is different in other countries). Sensitivity varies somewhat between different tests, but they are all roughly comparable and have the same strengths and weaknesses.
Epistemic status
This is a complex topic that is evolving quickly and is only partly understood. My analysis is grounded in hard data but necessarily involves a certain amount of extrapolation.
I have no relevant credentials but this writing has been reviewed by a medical epidemiologist who works full time on COVID.
Application 1: risk reduction
I consider antigen tests to be most useful for reducing the risk of asymptomatic transmission at social events. In that context, I believe a negative BinaxNOW test reduces the probability that you are infectious by about 50% for the 6 hour period immediately after taking the test. (There's no hard data behind the 6 hour cutoff—it's just a reasonable extrapolation based on what we know about viral load during the early stages of infection, and based on early data about Omicron).
When I host social events, I calculate the microCOVID risk of attending the event and include it in the invitation. At events where everyone tests at the door, I reduce the calculated risk by 50%. (Note that this is a rare case where you care about the sensitivity of a test, not the PPV).
Application 2: if you have symptoms
Home antigen tests have limited value for testing yourself when you have symptoms because their sensitivity is fairly low (probably about 70% for people with symptoms). I agree with current CDC guidance for people with symptoms (the guidance is in the middle of changing and as of mid October some documents are out of sync with others):
Option 1 is to take a home antigen test. If the results are positive, you probably have COVID: isolate and consider seeking medical advice. If the results are negative, you should still get a PCR test because of the substantial chance of a false negative.
Option 2 is to skip the home antigen test and get a PCR test right away.
A reasonable but sub-optimal third option is to isolate immediately and take multiple antigen tests, spaced 36 - 48 hours apart.
Application 3: testing after exposure
Current CDC guidance is that if you’re vaccinated and have been in close contact with someone who has COVID, you should get a PCR test 5-7 days after your exposure. Until then, you should wear a mask when you’re around other people indoors.
As with testing when you develop symptoms, antigen tests have limited value when testing after a known exposure. A positive test indicates you likely have COVID and further action is warranted, but a negative test is not super informative. (By the way, I like The microCOVID Project's blog post about negative test results).
My personal inclination (which is shared by my epidemiologist consultant) is to quarantine and perform serial antigen testing (see below) after a mild exposure and to get a PCR test after a serious exposure.
Application 4: serial testing
The final and somewhat niche application for antigen tests is serial testing, which is typically used by people who have a high degree of ongoing exposure or are highly risk intolerant. It typically involves testing every three days. Testing every day is not unreasonable, but testing more than once per day has very little value.
The idea behind serial testing is that if you’re testing regularly, one of your tests will occur soon after your viral load increases, warning you about an infection before you get severely ill or spread it to many people.
Serial testing is far from perfect, but early data suggest it can substantially reduce forward transmission and can achieve total sensitivity almost comparable to PCR testing. I’m not aware of any data or modeling of how much serial testing reduces forward transmission: if you know of any, I’d love to see it.
Using the BinaxNOW test
The Abbott BinaxNOW is a home antigen test for COVID that is widely available without a prescription and costs about $12 per test. It yields results in 15 minutes.
Using the test isn’t rocket science but it’s easy to make mistakes that significantly affect test accuracy. I recommend reading the instructions carefully the first time you use one (you might also watch this video). If you’re testing multiple people (at a dinner party, for example), you might consider having a designated person help everyone test and watch for any mistakes.
Common mistakes
Based on one published study and my own experience helping with numerous tests, I recommend you pay particular attention to:
A sample protocol
Here’s my protocol for events like dinner parties:
How accurate is BinaxNOW?
Short answer: BinaxNOW has an excellent specificity of 99%. Sensitivity is middling, with wide variation depending primarily on viral load. Those characteristics make it more useful for some applications than others: in particular, it’s more useful for determining whether someone is likely to be infectious than it is for determining whether someone has COVID at all.
Unfortunately, there is a lot of data but there isn’t a lot of high quality data that answers exactly what we want to know.
If you’re testing to find out if you have COVID, the overall sensitivity of BinaxNOW based on meta-analysis is:
All patients: sensitivity = 62%
All the antigen tests perform much better in people with symptoms. I haven’t found a meta analysis of this for the BinaxNOW specifically, but my best extrapolation of multiple data points is:
Symptomatic people: sensitivity = 67%
Asymptomatic people: sensitivity = 48%
If you’re testing to find out if you’re infectious, it’s a little more complicated. My best guess is:
Testing to see if you’re infectious: sensitivity = 50%
But you probably wouldn’t be here if you just wanted the short answer.
Data sources
I’ve found three papers to be most useful: this meta analysis (paper 1) from August 2021 provides the most comprehensive review of the available data, while this one (paper 2) and this one (paper 3) include subgroup analyses which are helpful for understanding what factors affect test accuracy.
Researchers generally determine accuracy by comparing BinaxNOW results to PCR results (the “gold standard”). Most studies used real-world testing of actual patients, but there is some data that uses lab-prepared samples (which is useful for understanding the underlying processes).
Subgroup analysis
The meta analysis found an average sensitivity of 62%, but that varied substantially between different subgroups. Three different subgroup analyses all suggest that sensitivity is highly dependent on how much virus is present—taken together, they strongly suggest BinaxNOW will be pretty good at detecting people who are currently infectious, but not so good at detecting low-grade infections or infections before or after peak viral load.
Symptomatic vs asymptomatic
Many studies have found better sensitivity in symptomatic people than asymptomatic. The meta-analysis (paper 1) found that for antigen tests in general, sensitivities were 72% and 52% in symptomatic and asymptomatic individuals. Extrapolating from other data about the BinaxNOW specifically, I’d guess:
Symptomatic people: sensitivity = 67%
Asymptomatic people: sensitivity = 48%
67% sensitivity isn’t great if you’ve just developed symptoms and you want to know if you have COVID or not.
Culture-positive vs culture-negative
Paper 2 performed a very interesting subgroup analysis: they tried to culture virus from each specimen and compared the sensitivity of culture-positive specimens to culture-negative ones. They found:
All specimens:
Sensitivity = 64% (symptomatic) vs 36% (asymptomatic)
Culture-positive specimens:
Sensitivity = 93% (symptomatic) vs 79% (asymptomatic)
Sensitivity of 79% in culture-positive specimens is quite good: if I had to pick a single metric of how sensitive BinaxNOW is for detecting asymptomatic but infectious cases, it would be this one. Viral culture is complicated to perform (especially for nasal samples), but many epidemiologists consider it to be the gold standard for detecting infectious individuals.
Ct values
Subgroup analysis based on Ct values provides strong evidence for the importance of viral load in determining test accuracy and is roughly consistent across multiple papers.
Some background: PCR tests work by detecting viral nucleic acid in a specimen. The process involves multiple cycles of duplicating nucleic acid: with each duplication cycle, any nucleic acid in the specimen gets copied. This results in an exponential increase in the amount of nucleic acid. The test keeps going until either there’s enough nucleic acid to be detectable, or enough cycles have been performed that there clearly isn’t any nucleic acid to be found.
The number of cycles performed is referred to as Ct (Cycle Threshold). A lower Ct indicates much more nucleic acid was present in the original sample, so fewer duplication cycles were needed to reach the detection threshold. Ct is a very useful indicator of how much virus was present in a specimen. Unfortunately, however, Ct values are not standardized across labs: there’s no standard Ct value that indicates someone is probably infectious.
Multiple studies have found that sensitivity depends strongly on Ct. From the meta-analysis of all antigen tests:
Sensitivity = 94% (Ct <= 25)
Sensitivity = 38% (Ct > 25)
From paper 3, for BinaxNOW specifically:
Sensitivity = 100% (Ct 13-19.9)
Sensitivity = 79% (Ct 20-24.9)
Sensitivity = 13% (Ct 25-29.9)
Sensitivity = 8% (Ct 30-35)
These results provide very strong evidence that sensitivity depends strongly on viral load (and therefore that sensitivity will be high when someone is infectious).
Putting it all together
So there’s lots of data, and it’s all pretty consistent: multiple lines of inquiry strongly suggest that BinaxNOW sensitivity is strongly dependent on viral load. So what’s the actual sensitivity?
If you’re testing because you’ve developed symptoms, have had an exposure, or are conducting serial testing, you should use a sensitivity of 67% if you’re symptomatic or 48% if you’re not. Those numbers are extrapolated from overall BinaxNOW sensitivity (from meta-analysis), asymptomatic vs symptomatic sensitivity across all antigen tests (from meta-analysis), and a study that measured asymptomatic vs symptomatic sensitivity in BinaxNOW specifically.
What if you’re testing to see if you’re infectious? That’s more complicated. The data are all pretty consistent, but nobody has directly measured what we want to know (because that would be very hard). The most directly relevant number is from paper 2, which found 79% sensitivity in asymptomatic people with culture-positive specimens.
Omicron update: I'm discounting from 75% to 50% until we have better data So I’m gonna pick 50% because it’s a round number—my gut says the real number might be a little higher, but for this application I think it’s appropriate to be a bit conservative.
Other bits and pieces
Performance with Delta and other variants
Paper 3 found that BinaxNOW seems to perform equally well with the Alpha and Delta variants, which isn’t terribly surprising. The paper found comparable performance across variants based on Ct values: given that Delta produces much greater viral loads, one could speculate that sensitivity with Delta might actually be superior (but without data, that’s purely speculation). Note, though, that (as with most COVID data), we still have limited data that is specific to Delta.
Stacking tests
People sometimes wonder if they can get better sensitivity by taking multiple tests at the same time. There is limited data on this, but multiple same-day tests seem to add almost no sensitivity.
The primary determinant of test sensitivity seems to be viral load: if you’re shedding a lot of virus the test is quite sensitive, and if you aren’t shedding much virus the test isn’t very sensitive at all. So if you’re not shedding much virus, the test isn’t very sensitive no matter how many tests you take in a row.
A minor factor in test accuracy is user skill, but rather than trying to correct for that by taking multiple tests, I’d recommend just reading the instructions carefully and making sure you’re doing it right.
BinaxNOW versus other tests
All the home antigen tests seem to have roughly comparable accuracy: variations between studies of the same test seem to be about as large as variations between tests.
Paper 1 found overall sensitivity of 71% for all antigen tests and 62% for the BinaxNOW specifically. And paper 3 found similar results between BinaxNOW, Quidel Sofia2, and BD Veritor.
As of October 2021, I think your choice of test should be driven by cost, availability, and ease of use more than accuracy.
Interpreting test results using Bayes factors
I like mayleaf's post on interpreting COVID test results using Bayes factors. Maybe you will too.
Other sources
There are lots of papers on this topic. Here are a few of my favorites:
The authors of paper 1 maintain a website that tracks all papers about antigen tests. It’s a great source if you want to do a comprehensive overview.
This September 2021 paper is a great meta-analysis of antigen tests but unfortunately doesn’t break out BinaxNOW specifically. Its results are roughly in line with the findings of paper 1, however.
An interesting modeling study that concludes test frequency and turnaround time are more important than accuracy.
A study of serial testing that finds good overall performance.