We are thinking about test performance all wrong. Here's a better way

Whether it's a PCR, a LAMP assay, antigen detection, or viral culture, the first question experts ask is how sensitive (which describes how many truly positive cases a test fails to detect) and specific (which describes how many positive tests are misleading) a new SARS-CoV-2 test is. Fears have promulgated around the idea that some tests miss up to 30 percent of cases. 

That's the wrong way to look at it, argues a new Perspective published Thursday in The New England Journal of Medicine. In a compelling essay, the authors state that we have to rethink testing entirely in order to get around these fears while not missing contagious cases. The insight: stop thinking of tests as one-offs and stop performing tests as one-offs. Instead, testing regimens should be evaluated. 

If a particular at-home test fails to detect an active case 20 percent of the time, then an alternative testing regime for that same product must be evaluated to see how often it would miss cases when, for example, three consecutive tests are performed (perhaps in a short window, or perhaps over several hours). In this example, if one in five tests were falsely negative (i.e. the person has covid-19 but the test is negative) simply due to chance—owing, say, to inexpensively engineered disposable home test kits—the odds of three negative tests in a row in a person who is actually infected would be under 1 percent. Suddenly an inexpensive test that might have been criticized for being too inaccurate ("missing 20 percent of infections") might instead be celebrated. Therefore, such tests should be vetted not based on their one-time performance, but instead as a well-defined regimen. 

None of this should be that foreign a concept. A similar approach is already used by physicians, researchers, and scientists in other areas of public health and medicine, argue the authors. We do not judge an antibiotic by how well it works after one dose. We evaluate the therapy based on multiple doses—regimens which were developed with this exact framework in mind.

To achieve this vision, the idea of one-and-done testing must be discarded. Instead, we need massive investments in technologies that will allow us to implement testing regimens at home and in medical settings. The beauty of this approach is that different tests can be used in different situations, once the performance accuracy of various regimens have been determined. For example, many people test positive for SARS-CoV-2 after they have already passed their contagious period (perhaps half of all infections, it has been suggested). For these people, a 10-to-14 day isolation period would be excessive, if it could be confidently shown that they are outside the contagious period. Conversely, testing too early misses cases that are on the verge of becoming contagious. A validated test regimen that addresses this (using a regimen of inexpensive repeated home testing, for example) could save many lives. The problem is that individual tests have been approved by regulatory agencies, not testing regimes. As we have learned, individual tests that are inadequately sensitive (or used at the wrong time) can be very dangerous. In contrast, a multi-step well-defined testing regimen using some of the same tests already on the market could both detect the early contagious window in some infected individuals and reassure those well beyond that phase.


PPE problems persist. How is the Strategic National Stockpile looking?

Personal protective equipment (PPE) shortages have been the constant background of the pandemic in the United States. As early as April, Brief19 reported on healthcare workers becoming infected due to shortages. In the absence of a unified federal plan, grassroots movements like #GetUsPPE (now Get Us PPE) gained substantial attention. It and other organizations repeatedly petitioned the President to invoke the Defense Production Act to prioritize increased manufacturing, while federal agencies gave warning that demand for masks, gloves, gowns, face shields, and other needed items continued to outpace supply. In September, the Food and Drug Administration (FDA) published a list of PPE-related supplies in shortage.

Against this backdrop, a new investigation by National Public Radio (NPR) has found that the Strategic National Stockpile (SNS) is still unable to keep pace, citing budget shortcomings, lack of domestic manufacturing, and a global supply chain still in chaos.

The SNS was designed as a stopgap in the face of an overwhelming national or global crisis (of which a pandemic is merely one), meant to be able to provide up to ninety days of supplies during the initial response to a wide range of chemical, biological, or nuclear incidents. Representatives from the Department of Health and Human Services (HHS) list 142 million N95 masks and 22 million pairs of nitrile gloves, of the projected 300 million and 4.5 billion required to fill the stockpile.

Procurers state that many of these necessary prerequisites are manufactured in very few places, and without a domestic source (including those that would be made possible by the invoking of the Defense Production Act), it has been a veritable global auction house to find these supplies. Absent a coordinated effort from the federal government, state and local health entities have been forced to enter this chaotic system and fend for themselves in trying to meet the ever-present need to protect our healthcare providers. But those jurisdictions lack the financial resources needed. And unlike the federal government, they can't just print money to fund these initiatives. National Public Radio.

  • "Any Given Thursday." How many Americans became asymptomatic transmitters today?
  • Vaccine efficacy versus effectiveness
  • Brief19's Outbreak Data Tracker

  • Trial of convalescent plasma for covid-19 falls flat, another setback for hyped treatment
  • US pediatric testing results for covid-19: The kids are...just ok. And not immune