September 26th, 2013

Scrutinizing the PRAMI Trial of “Preventive Angioplasty”

Victor Montori offers his analysis of the PRAMI trial, recently published in The New England Journal of Medicine.


In a patient-blind trial, 465 patients with STEMI and multivessel CAD were randomized to undergo infarct-artery–only PCI or additional PCI in non-infarct arteries during the initial procedure. Patients with cardiogenic shock, prior coronary artery bypass grafting, significant left main disease, or chronically occluded arteries were excluded. The study was stopped early (mean follow-up, 23 months), when a significantly lower incidence of the primary composite endpoint — cardiac death, MI, or refractory angina — emerged in the group that received the additional (non–infarct-artery) PCI compared with the group that underwent infract-artery–only PCI (hazard ratio, 0.35; 95% CI, 0.21–0.58). Rates of procedure-related complications were similar in the two groups.


PRAMI has garnered attention for showing that, in patients with STEMI and multivessel CAD, stenting culprit and nonculprit lesions (“preventive” PCI) reduced the risk for the trial’s primary composite endpoint, compared with stenting only the culprit lesion. Most discussions of this trial have focused on its findings rather than its methods. The crucial question to be addressed: What should our confidence in this estimate of effect be, given that, as the article notes, “By January 2013, the results were considered conclusive by the data and safety monitoring committee, which recommended that the trial be stopped early”?

As my study group has found, stopping trials early because of an unexpectedly large treatment effect is a sure way to overestimate that effect, particularly when the number of events is low. The overestimate because of truncation is small after 500 outcomes, moderate for 200 to 500, and large for <200 events. We have also found that stopping trials early increases the trial’s chances of being published in a top-tier medical journal and of receiving rapid dissemination and incorporation into guidelines. The interpretational challenges increase when the trial is stopped on the basis of the effect of therapy on a composite endpoint: Stopping early guarantees an imprecise assessment of the effect of therapy on the infrequent — and often more important — outcomes that make up the composite endpoint.

The PRAMI trial illustrates all of these points. First, it was stopped after only 74 outcomes had accrued. Second, despite its size, the trial was published in the NEJM. Third, the sparse events were distributed in the components of the composite endpoint that differed significantly in their frequency — cardiac death (14 events), nonfatal MI (27 events), and refractory angina (42 events) — and in their importance to patients. Also, the precision of these estimates and the accompanying P values are extremely sensitive to the addition of just a few events. How many more MIs would need to have occurred in the preventive PCI arm to render the effects on nonfatal MI (P=0.009) nonsignficant? Three. Just three.

One might argue that we should not worry too much about these small trials, given that they can later be pooled in meta-analyses. Our group has shown that such an exercise is fraught with problems: Trials stopped early tend to carry undue weight in meta-analyses because of the effect of publication bias of negative trials. In addition, trials testing the same question are delayed because of the assumption that it is no longer ethical, for example, to randomize patients to not undergo preventive PCI. As a result, the trials that are stopped early gain even more weight in meta-analyses.

Recall that PRAMI’s data and safety monitoring board determined that it was no longer appropriate to continue the trial as planned. So how does one now justify further confirmatory trials? This a false dilemma: The duty to protect people in the trial cannot exceed the duty to protect the much larger population that could be exposed to a potentially harmful intervention that has been supported by an inflated estimate of effect. The imprecise and potentially overestimated results of PRAMI must be tested. It is feasible, ethical, and necessary — and it should have appeared as such to the data safety monitoring board.

The confidence in the estimates coming from PRAMI should be tempered accordingly, to account for the factors described above. What should the researchers have done to prevent this loss in confidence in the results of their trial? They should have decided not to introduce efficacy-stopping rules. If that was not feasible, they should have set up stopping rules that would be triggered only after a large number of outcomes have accrued.


Do you agree with Dr. Montori’s analysis of the PRAMI trial?

Click here for a previously published interview with PRAMI’s lead investigator.

5 Responses to “Scrutinizing the PRAMI Trial of “Preventive Angioplasty””

  1. Yadon Arad, MD says:

    Absolutely agree with Dr. Montori’s analysis. No clinical trial should be published before going through a similar analysis as well as accounting for multiple endpoints and other statistical issues. There is also the issue of publication bias towards positive results. I doubt this study would have been published in NEJM if the results were negative. My guess is that the reviewers would have used these and similar arguments to reject the study. Perturbation analysis should always be done.
    Yadon Arad MD FACP, FACC, FACE

  2. Isaac Vilayil Mammen, M.D., D.M. says:

    Why did they stent all lesions more than 50% when the guidelines states 70% for non LMCA lesions? Should we start stenting 50% lesions also ??

  3. Outstanding commentary. Provocative work. More research needed.

  4. David Powell , MD, FACC says:

    I agree. I was surprised to see this article in the NEJM without an editorial more strongly addressing the immense methodological problems. Is there not a statistical society that sets criteria for stopping rules? I thought that the rules were based on Bayesian principles? Even if they were, the non double blinded nature of the trial clearly affected the frequency of stress testing in each group after randomization. This foreseeable confounding factor should have motivated a much larger trial and/ or longer follow-up with less weighting on soft endpoints . They should provide the patient level data as well.

  5. Felix Valencia, MD, PhD says:

    After letters to the editor publication in NEJM, we still don’t know:
    -Ejection fraction or time to reperfusion among groups, making difficult the interpretation of the overall results of the trial.
    -We don’t know if any pt underwent fybrinolisis, it was not an exclusion criteria. Moreover, primary PCI was not an inclusion criteria. In fact the term applied all over the paper is emergent PCI, avoiding carefully naming trial procedure as primary PCI. It’s obvious that this make a point when interpreting the results. In fact, primary PCI and rescue PCI are emergent PCI procedures in STEMI with different outcomes and prognosis.
    -Finally, did the trial results were largely driven by the results in the “control” group? Pts in the control group were in an ultraconservative strategy: non culprit lesion were only treated when refractory angina was present and evidence of ischemia in non invasive test was demostrable. This strategy in post MI pts with multivessel disease and with a clear indication for revascularization per se (refractory angina and known coronary anatomy suitable for PCI) may explain why patients in the control group did worse than those in the intervention group.