January 10th, 2011
Have the COURAGE to Critique a Substudy
Harlan M. Krumholz, MD, SM
In this journal club, I compare the published data from an original trial with the authors’ conclusions in a substudy from that trial. Often, a substudy provides valuable insights that complement the initial trial findings. Sometimes, however, you need to look closely to identify the additional insight. Case in point: COURAGE.
The Original COURAGE Trial
As reported in 2007 in the NEJM, 2287 patients with objective evidence of myocardial ischemia and significant epicardial coronary artery disease were randomized to receive either optimal medical therapy (OMT) alone or percutaneous coronary intervention (PCI) plus OMT. During a median follow-up of 4.6 years, incidence of the primary endpoint — death or nonfatal MI — was statistically similar in the two groups, but slightly higher with PCI (OMT alone, 18.5%; PCI, 19.0%; P=0.62). PCI also showed no advantage in the individual endpoints of death, nonfatal MI, or hospitalization for acute coronary syndrome — nor (as reported subsequently) in quality of life.
The Nuclear Substudy
In 2008, the COURAGE nuclear substudy (the first substudy from the trial) was published in Circulation. It involved 314 patients who underwent serial rest/stress myocardial perfusion single-photon-emission computed tomography (MPS), both before treatment and then 6 to 18 months after randomization. The primary endpoint was defined as ≥5% reduction in myocardial ischemia at follow-up. The investigators report that the PCI group had a significantly greater mean reduction in ischemic myocardium (–2.7% with PCI vs. –0.5% with OMT alone) and a significantly larger percentage of patients who achieved the ≥5% reduction endpoint (33% vs. 19%). The ≥5% reduction in ischemic burden was not significantly associated with better outcome in a multivariable analysis, and as the authors acknowledge, their study “was not powered to examine differences in clinical outcomes according to change in ischemic burden.” In the conclusion of their abstract, the authors say that “adding PCI to OMT resulted in greater reduction in ischemia compared with OMT alone.” They further state, “Our findings suggest a treatment target of ≥5% ischemia reduction with OMT with or without coronary revascularization.”
The Leap in Logic
The authors’ conclusion is quite provocative. The full trial showed that patients with ischemia did equally well with an initial strategy of PCI or OMT. In the substudy, the authors assert that, given their results, clinicians should seek a target of ≥5% ischemia reduction. The implications are that these patients should undergo MPS, that therapeutic strategies should be guided over time by the MPS results, and that PCI is better than OMT alone in reducing ischemia. How did the authors make this leap? Their reasoning appears to be as follows:
The PCI strategy was better than OMT in reducing ischemia burden by ≥5%. The authors place emphasis on a significant unadjusted association between a reduction in ischemic burden and a reduction in risk (but a critical reader will note that the multivariable analysis was not significant). On the basis of the unadjusted result, the authors conclude that reducing ischemic burden by ≥5% should be a treatment goal and, by implication, that PCI plus OMT would be better than OMT alone for that purpose.
Untangling the Logic
The substudy was not designed to test whether targeting treatment to a certain threshold of ischemia reduction benefits patients; it simply suggested that the PCI strategy was more effective than OMT alone in reducing ischemic burden. [Given that the subjects in this substudy represented a convenience sample of patients and the two treatment groups were not similar it is not clear that they have even demonstrated that PCI reduces ischemic burden more effectively – this is not a comparison of groups that were obtained by randomization – a nuance that is acknowledged but may be easy to miss – and they do not do what is necessary to control for the differences.] Notably, even if PCI reduced ischemic burden, PCI did not significantly reduce the risk for clinical events or substantially affect symptoms in the overall trial. Moreover, in the substudy,the reduction of ischemic burden also did not reduce the risk for events (according to the multivariable analysis in the substudy, which was important since those who had a reduction were likely different at baseline from those who did not – but they do not present this comparison). If ischemic burden reduction had really mattered, the trial should have shown evidence of clinical differences between the two treatment groups.
Another issue is that ischemic burden is a surrogate for the clinical outcomes that affect patients. Surrogate endpoints can be useful when we do not have clinical outcomes to examine. In COURAGE, clinical outcomes were reported. Perhaps the most important finding in this substudy is the failure of ischemic burden as a surrogate endpoint. If COURAGE had just used ischemic-burden reduction as the primary endpoint, the PCI advocates would have declared victory. However, the trial identified no difference between the groups in the major clinical endpoints that were measured, and the surrogate endpoint would have been useless in predicting the outcome of the trial. This finding suggests to me that treating to a target reduction in ischemic burden is not useful.
There are also other issues to consider regarding the study design. In order for the trial participants to have their ischemic burden tracked over time, they had to be around for the follow-up study. They are, then, patients who survived at least 6 months and, in some cases, up to 18 months. We do not have information about who in this substudy was lost to follow-up or did not return for a follow-up MPS study — and whether the time to the follow-up MPS was similar in both intervention groups. That information is essential to evaluating the substudy.
A Better Line of Inquiry
What nuclear substudy might have been most useful to a clinician? I personally would have wanted to know whether PCI was better than OMT alone among patients with a large ischemic burden at baseline. Such an analysis would have investigated an interaction — whether the difference between the PCI and OMT-alone groups varied according to baseline ischemic burden. A reasonable hypothesis would have been that the PCI strategy was superior for patients who initially had the most ischemia according to MPS. Unfortunately, the investigators could not do that analysis, probably due to lack of resources.
Do you think this COURAGE substudy complements the original trial? How do you assess the authors’ suggestion to target a certain reduction in ischemic burden for patients like those in this trial? How would you have written the conclusion to the abstract?
12 Responses to “Have the COURAGE to Critique a Substudy”
NEJM Journal Watch — Recent Cardiology Articles- Anticoagulation in Patients with Cancer-Associated Pulmonary Embolism: How Long is Long Enough?
- Spironolactone Use After Percutaneous Intervention for Acute Myocardial Infarction
- CRISPR/Cas9 Gene Editing for Transthyretin Cardiomyopathy
- Does Colchicine Reduce Cardiac Events After Myocardial Infarction?
- First-Line Therapy for Ventricular Tachycardia After MI: What's the Best Approach?
I agree, not a robust study,..but does point towards a possible marker to foster future clinical trials….This should pave the way for a simple trial of moderate or greater ischemic burden (>10% TPD) on imaging randomized to OCT vs. PCI, and assess again for hard endpoints, along with reduction in ischemic burden. Primary endpoint should still be MACE, though again, to power this, one will need a large group of patients. Interesting take on the studie’s interpretation of ischemic burden is that they used total perfusion defect rather than routine Summed stress score, that is more commonly used. If TPD is more predictive, this should be customized into various software algorithms for all nuclear reads for interpretation and standardization…Currently not the case.. I still find the study to be provocative..
There are 2 appropriate conclusions to be drawn from the COURAGE nuclear substudy.
1. An initial treatment strategy of PCI+OMT resulted in a statistically significant difference in ischemic burden (both mean and 5% categorical ischemia reduction criteria) compared with OMT. However, the clinical relevance of this difference is unclear.
2. Neither residual ischemia nor reduction in ischemic burden was independently associated with death or MI.
Guidelines recommend a 10% ischemic threshold for coronary revascularization in patients with stable angina and normal LV systolic function. This number is largely based on a single center registry of 10,647 patients (Hachamovitch et al, Circulation 2003) that showed survival benefit to be NUMERICALLY greater with revascularization (CABG + PCI) compared with medical therapy in patients with >10% ischemia (12% of cohort) and STATISTICALLY greater only in patients with >20% ischemia (5% of cohort). Even though the survival curves crossed at 10% ischemic threshold, there was substantial overlap in 95% confidence intervals of the survival curves until >20% ischemia.
Thus, a treatment goal of reducing ischemia burden by >5% (as suggested by the authors of the substudy), as well as recommending revascularization over medical therapy at >10% ischemia (as recommended by the guidelines) is not supported by valid scientific evidence.
The clinical importance of various ischemic thresholds in clinical decision making warrants careful evaluation.
Sanjay Kaul
George Diamond
George and Sanjay – do you even accept that this substudy showed that an initial treatment strategy of PCI+OMT was superior in producing a reduction in ischemic burden. Wouldn’t it be appropriate to adjust for baseline differences in the groups (this is not a comparison of groups generated by randomization) – and wouldn’t you want to know the mean time to the MPS study in each group?
This suggests that the sample in this nonrandomized substudy is selected and deviates from the overall COURAGE population.
Thus reads the last sentence BEFORE the” Conclusions”.
I appreciate HK’s courage to confront a major defense of those who minimize the Courage message.
Competing interests pertaining specifically to this post, comment, or both:
none
Harlan, we agree that a more principled analysis should adjust for baseline differences and time to MPS between the groups. However, despite the sample being nonrandomized and selective (lower risk than the main trial cohort), the pretreatment clinical characteristics appear to be well balanced across the groups (Table 1). “Given the P value of less than 0.0001 for mean ischemia reduction and P value of 0.0004 for greater than 5% ischemia reduction, it is unlikely that the risk- and time-adjusted results would be statistically inconclusive.”
Nonetheless, any claims of superiority (we were careful in avoiding the term) based on a nonrandomized comparison, especially when the main trial results were inconclusive, would be a case of overinterpreting the data. Hence our caveat “the clinical relevance of this difference is not clear”.
A greater issue is the latitude the guidelines (ESC/EACT guidelines on myocardial revascularization 2010) have taken in interpreting the COURAGE nuclear substudy results to support Class I, level B recommendation for revascularization in stable angina in patients with >10% ischemia. Guidelines that are not based on high-quality evidence could potentially misinform clinical practice and mislead future research.
Sanjay Kaul
George Diamond
Are you aware about any randomized trial going on dealing with this kind of patient (>10% ischemic burden)? I believe it would be critical for clinical practice.
Competing interests pertaining specifically to this post, comment, or both:
none
This is an excellent discussion. The purpose of the substudy was to be hypothesis generating, and unfortunately it has been misinterpreted by some to mean that PCI reduces death or MI in patients with >10% ischemia. I was surprised to see that the study was used as evidence to support PCI in the ESC/EACTS guidelines that misquote the findings as follows: “Most recently, in a small nuclear substudy of the COURAGE trial (which reported no overall survival benefit of PCI over OMT), involving just over 300 patients, 100 patients with >10% ischaemic myocardium had a lower risk of death or MI with revascularization.”
The NIH is currently reviewing a proposal for a large multicenter trial in stable patients with >10% ischemia to compare an invasive strategy, starting with cardiac catheterization followed by revascularization plus optimal medical therapy (OMT), with a conservative strategy of OMT, with cath and revascularization reserved for patients with refractory symptoms or acute cardiac events. In the meantime the COURAGE investigators are completing the analysis recommended above by Harlan: a comparison of outcomes in patients who had site-interpreted nuclear stress tests at baseline with at least moderate ischemia.
Competing interests pertaining specifically to this post, comment, or both:
none
David,
There was an abstract presented at the ACC Scientific Sessions in 2010 that reported there was a lower risk of death or MI with PCI + OMT compared with OMT in the subset of COURAGE patients who had >10% ischemia at baseline. Perhaps, the ESC/EACTS guidelines were based on these data. Are you implying that the data were not correctly interpreted? If so, what is the correct interpretation of the data? And what impact does it have on the hypothesis driving the NIH trial?
Thanks
Sanjay
Sanjay… on your previous comment – I am surprised that you so easily dismiss the differences in the groups – 3x as many high risk exercise studies in the OMT group – the OMT group exercises for more than a minute less time (that is a huge difference) – I don’t care about the P values – this selection here is unclear – and most importantly you do not know how the time to the MPS compares in the two groups… do you?
Harlan,
There is no disagreement with the overall thrust of your argument that the data do not permit an inference of superiority in ischemia reduction with revascularization. In a nonrandomized sample, there are bound to be some imbalances. It is unclear to me what is the clinical relevance of imbalances in pretreatment exercise or distribution of Duke treadmill score. It is interesting to note that the delta in exercise time (follow-up minus pretreatment) was similar in both groups, i.e., 1.2 minutes. Unless there was a systematic bias towards earlier follow-up MPS in the OMT group, it suggests that both treatment strategies were equally effective in improving exercise time. Nonetheless, the time to follow-up MPS data should be presented and its impact on outcomes explored.
Sanjay,
Sorry for the delay in responding. I’m new to this blog process.
It is clear that the ESC guidelines were not based on the presentation at the ACC Scientific Sessions last year because it (the abstract) is not referenced in the ESC guidelines and the numbers used in their text (n~300) is the number in the substudy we published in Circulation in 2008 (which is referenced in the ESC guidelines).
I am not implying, but rather stating as plainly as I can, that the data published in Circulation were not correctly interpreted in the ESC guidelines. I believe the correct interpretation is that although there was significantly greater reduction in ischemia by PCI+OMT as compared with OMT alone, particularly in patients with >10% ischemia, there was no significant (risk-adjusted) reduction in events associated with the reduction in ischemia. There was, however, a provocative signal suggesting that with a sufficient sample size a significant reduction in the composite endpoint of death or MI might be demonstrated. The impact of that interpretation on the hypothesis driving the NIH-proposed trial is that we need a sufficiently powered trial to prove the validity of the current guidelines that recommend revascularization in stable patients with substantial ischemia.
David
Harlan is to be appreciated for bringing out the facts