COVID Platform Trials Called Into Question

While efficient, the efficacy of international platform trials in getting sound results on COVID-19 was called into question by NIH statisticians.

This type of randomized clinical trial (RCT) skips a lot of the time-consuming work of setting up separate trials. Instead, multiple experimental treatment groups are compared with a single control group and experimental arms can enter and exit the trial at different times as evidence emerges.

Large pragmatic platform trials have been behind most of the successes in finding treatments for COVID-19, including the RECOVERY and REMAP-CAP trials.

However, this feature that leads to the speed useful in a pandemic can also undermine the key tenet of RCTs: that randomization yields equivalent patient populations so that between-group differences can be attributed solely to the treatment.

Nonconcurrent control groups can be a problem in a fast-changing clinical environment like COVID-19, argued Lori E. Dodd, PhD, of the National Institute of Allergy and Infectious Diseases, and Boris Freidlin, PhD, and Edward L. Korn, PhD, both of the National Cancer Institute, all in Bethesda, Maryland.

"Consider the decline in in-hospital mortality from Covid-19 that occurred over a 2-month period in the spring of 2020 and a hypothetical trial that compared a control treatment with an ineffective new agent that was not included in the randomization until the second month," they posited in a letter to the editor in the New England Journal of Medicine (NEJM).

"If comparisons were made between the patients who received the control treatment during the 2-month period (April-May 2020) and the patients who were randomly assigned to receive the ineffective new agent over the 1-month period (May 2020), the results would erroneously suggest that 30-day mortality would be 37% lower with the ineffective new agent," they noted.

Nonconcurrent control groups can create bias, Dodd's group said, when randomization ratios between trial groups change over time, when countries or sites with different mortality rates restrict randomization into the control group or to certain trial treatments, or when concomitant treatments differ between trial treatment groups.

In their letter, released in conjunction with the print publication of the REMAP-CAP and COVACTA trial findings on interleukin-6 inhibitors for severe COVID-19 (following online release in February), the trio called out REMAP-CAP in particular for such bias in favor of tocilizumab (Actemra) and sarilumab (Kevzara).

"In this international platform trial, the control group used in the analysis was not restricted to patients who had undergone concurrent randomization, and both trial agents were reported to improve survival," Dodd and colleagues wrote.

"One cannot say with certainty that the statistical modeling was not successful in eliminating bias, especially in a complex and hard to understand platform trial such as REMAP-CAP," they concluded. "The added value from this trial relative to the other randomized trials with straightforward, comparable controls can be questioned."

REMAP-CAP's senior statistician, Scott Berry, PhD, argued, though, that this group of authors with a history of pushing back on statistical innovations was throwing "misplaced shade."

"It's appropriate to ask these questions," he told MedPage Today. "What I was disappointed with was them attaching it to the results of the REMAP-CAP trial on tocilizumab and sarilumab, because they don't have this issue."

"Maybe we didn't necessarily explain this well, but we presented exactly the analysis they wanted, which was only concurrent controls," he noted, adding that "the results in concurrent controls were actually slightly better."

In fact, every single paper from REMAP-CAP that's coming out does the same, Berry said.

He has been involved in the design of other large platform trials, like ATTACC and some arms of the ACTIV platform, and argued that the NEJM letter should not undermine clinicians' trust in the results.

"The platform trials have presented incredibly reliable results," he said. "Most of what we know about how to treat people [with COVID-19] has come out of platform trials. It's what the NIH Operation Warp Speed created with its ACTIV program, and it's really been quite effective."

Communication is key to maintaining that trust, commented Behnood Bikdeli, MD, of Yale's Center for Outcomes Research and Evaluation in New Haven, Connecticut, who has been involved in designing COVID-19 clinical trials like INSPIRATION.

"Final results and their tradeoffs compared with traditional trials should be communicated in a transparent understandable way to the medical community, many of whom may not have detailed knowledge of clinical trial design," he told MedPage Today. "That said, before all the details are published, it is difficult to fully understand the strengths and limitations of the results. I remain enthusiastic to see the full results of the multiplatform trials."

Comment