RESEARCH BRIEFING

Two high profile journals issue retractions

The New England Journal of Medicine and The Lancet retracted two covid-19 research articles yesterday. Both papers had been primarily written by Dr. Mandeep Mehra of Brigham and Women's Hospital in Boston. The data used in both studies came from a previously little-known company called Surgisphere, led by Dr. Sapan Desai. Surgisphere has marketed itself as "#1 in Machine Learning-Powered Data Analytics."

On May 1, NEJM published a paper entitled "Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19." In the study, using data provided by Surgisphere, the authors concluded that there no association between the use of certain blood pressure medications (ACE inhibitors and "ARBs") and the risk of dying in the hospital amongst covid-19 patients. These medications had been theorized by some experts as possibly dangerous to covid-19 patients. While the findings of the paper bolstered the opinions of those who believed the medications to be safe for covid-19 patients with pre-existing high blood pressure, the paper largely flew under the radar.

On May 22, The Lancet published a paper, covered on Brief19, that also relied on Surgisphere data and was primarily authored by Dr. Mehra. The study assessed outcomes of covid-19 patients admitted to hospitals who were given the drug hydroxycholorquine. This paper had a blockbuster result: approximately 35 percent of patients who received hydroxychloroquine or chloroquine (with or without the antibiotic azithromycin) died in the hospital compared to only 9.3 percent of patients who had not received these medications. While other prominent observational studies had suggested similar findings, the numbers were decidedly less dramatic, and far fewer patients had been assessed. Due to its size, 96,032 patients, and the large number of deaths in the patients who received hydroxychloroquine, the Lancet results made headlines. The World Health Organization temporarily halted its large randomized clinical trial in response.

But experts started asking how a company no one had heard of could have data from 671 hospitals across six continents. Researchers in Australia noticed that the Surgisphere database had more covid-19 deaths reported in the Lancet study than the public health department had reported for the entire country. Researchers noted that it was highly unlikely that hospitals in Africa that covid-19 deaths had the infrastructure to provide such detailed data to Surgisphere. Additionally, the average age of patients was almost 10 years younger in the Surgisphere database than those reported from most countries. In response, a revised version of the paper was published with updated results last week. The authors claimed the discrepancies stemmed from a misclassification of an Australian hospital that should have been allocated to Asia. But the update only raised more questions. Some data in the first version had changed in the revision.

By then, a flurry of skeptics had emerged, tearing the paper apart, line by line. A group of health services researchers took to Twitter, claiming that "supplemental data" in an appendix to the article did not appear to make sense. For example, data on race was reported for the entire cohort of patients but was noticeably missing from data describing averages of entire continents. It also seemed unusual that all six continents would code race similarly to how we code race in the United States. Other researchers tried modeling data to see if the results were even feasible. In so doing, they found numerous discrepancies which undermined the credibility of the results.

After NEJM and The Lancet issued "expressions of concern" regarding the validity of the data earlier this week, Dr. Mehra launched an audit of the data. Mehra later told Science Magazine "independent peer reviewers informed us that Surgisphere would not transfer the full dataset, client contracts, and the full ISO audit report to their servers for analysis as such transfer would violate client agreements and confidentiality requirements," which made the independent audit of the data impossible. "Based on this development, we can no longer vouch for the veracity of the primary data sources." NEJM and The Lancet then quickly retracted both publications, sooner than many had expected. The WHO's major international trial that had halted its recruitment of patients last week announced plans to resume.

Questions remain. In both papers, Dr. Mehra wrote that he had access to the data and took full responsibility for it. He told Science Magazine that in the rush to publish during the covid-19 crisis, "I did not do enough to ensure that the data source was appropriate for this use. For that, and for all the disruptions—both directly and indirectly—I am truly sorry." But it remains unclear whether Mehra ever saw the raw data, or merely summaries that Surgisphere provided. According to research standards, however, it is likely that Mehra signed documents stating that he saw and controlled the entire dataset when he submitted the papers to NEJM and Lancet. If he signed such documents but did not see the raw data, his apology would have to be expanded.

Research Section Editor

POLICY BRIEFING

Making sure poor countries can get a SARS-coV-2 vaccine

Poor and middle-income countries are likely to be hardest hit in the current pandemic, given their less robust public health systems and conditions that promote the spread of the virus, like high-density housing. But when a SARS-coV-2 vaccine comes to market, those countries will have less buying capacity than wealthier ones. To address this problem, a public-private partnership called GAVI — backed by the Gates Foundation, the World Health Organization, the World Bank, UNICEF and governments of countries around the world — is proposing to make something called an advance market commitment. Under that mechanism, GAVI would commit to buying a minimum number of vaccines at an established cost for low and middle income countries. This guaranteed purchase would eliminate the risk of poor demand or inability to pay for the manufacturers of the vaccines. The idea of incentivizing development of drugs and therapeutics that may not be financially lucrative is not new; the Food and Drug Administration's orphan drug program currently does this, for example. An example of a similar proposal is the Health Impact Fund (HIF) which was proposed as a World Trade Organization mechanism nearly a decade ago but has not been implemented. Under the HIF, if companies would commit to making a drug available to low-income countries for the lowest possible price, they would be eligible for an annual award from the WTO, proportionate to the drug's health impact. However, HIF would not address a country's ability to pay, even a very low cost. The advance market commitment mechanism, which GAVI has used previously for pneumococcal vaccine, would both incentivize the process of developing vaccines and ensure equitable access to them.

Policy Section Editor

Privacy protection bill introduced

Contact tracing applications are being developed by technology companies to better understand the spread of the coronavirus. With these apps come associated concerns about personal data protection. In response, Senators Maria Cantwell (D-WA) and Bill Cassidy (R-LA) have introduced the Exposure Notification Privacy Act. To promote legitimacy, the bill would require that public health officials be involved in the deployment of any apps and that only medically-authorized diagnoses be included.  From the standpoint of personal privacy best practices, any developed technologies would explicitly need to be opt-in and users would need to be allowed to delete their data from the system at any point. In addition, users would not be permitted to be discriminated against in places of public accommodation if they chose not to opt in.  Finally, any data gathered would not be allowed to exceed the minimum amount necessary for the system to function and data could not be used for commercial purposes. The law would also require comprehensive data security and breach notification, as well as creating a method for enforcement of any violations. The US Senate.