July 26th, 2010

Questions for Sanjay Kaul about TIDE and Avandia

and

CardioExchange’s editor-in-chief Harlan Krumholz discussed the TIDE trial on email with Sanjay Kaul, who was a member of the FDA’s advisory panel last week on Avandia. Here is a lightly edited version of their exchange.

Krumholz: What is your response to the FDA announcement that it has placed TIDE on a “partial clinical hold”? Do you think the trial should be completed? Some have said the trial is unethical.

Kaul: I am not quite sure what to make of it. The FDA should conduct a 100% audit of the RECORD trial data to gain reasonable assurance of the credibility of the results. Just like the JUPITER trial, in which enrollment got a boost when the FDA rejected the Public Citizen’s Health Research Group and David Graham’s petition to pull rosuvastatin from the market,  if the FDA clears the RECORD trial of any misconduct or malfeasance, it is likely to have a positive effect on TIDE enrollment.

If so, the TIDE trial should be allowed to continue with informed consent and design modifications. Informed consent should appropriately reflect the concerns about cardiovascular risk with rosiglitazone without using language that would discourage subjects from participating in the trial, thereby rendering it futile — an ethical dilemma as challenging as that posed by claims of lack of equipoise between rosiglitazone and pioglitazone.

Regarding design modification, one proposal is an adaptive design using the current data to reassess power and sample size calculations. In addition to the primary hypothesis of superiority of TZDs (either rosiglitazone or pioglitazone) or noninferiority of rosiglitazone compared with placebo when added to sulfonylurea (SU), metformin (MET), or both, a co-primary hypothesis should compare rosiglitazone with pioglitazone.

The key question we face is whether the quantity and the quality of the available evidence is sufficient to adjudicate the cardiovascular risk of rosiglitazone. Based on the quantity of evidence, a small-to-modest risk is associated with rosiglitazone. However, when one looks at the quality of data informing this judgment, most are of a caliber that do not, in my opinion, allow reliable causal inferences. For example, epidemiologic studies have inherent limitations; meta-analyses have yielded inconsistent results that are not sufficiently discernible from a null effect (I do not consider a meta-analytic P value of <0.05 to be strong evidence, and meta-analyses are mostly good for asking questions, not answering them); post hoc analyses of observational datasets derived from large randomized controlled trials have limitations that challenge their interpretability; and, finally, the open-label, noninferiority, randomized controlled RECORD trial was not optimally designed and conducted and therefore not well suited to meaningful inferences.

Given the limitations of the existing data, no clear and convincing evidence either incriminates the drug or exonerates it. Thus, I feel that it is in the best interest of science that the highest-quality evidence obtained from well-controlled randomized trials be allowed to confirm or refute the CV risk signaled by lower-quality data.

Krumholz: But for my edification, Sanjay, why would anyone participate in this trial? Am I missing something? What is the potential upside for any patient to participate in a safety trial? I worry that the only people being enrolled are those who are desperate for free meds.

Kaul: With the possible exception of metformin (UKPDS), no glucose-lowering drug has been shown to reduce CV outcomes. So it is worthwhile to evaluate whether TZDs alter CV risk. TIDE is a large, double-blind RCT to determine whether TZDs reduce CV and other serious health outcomes and whether rosiglitazone and pioglitazone have similar or different effects.

If I viewed TIDE purely as a safety trial, with no conceivable benefit to patients, then I could be persuaded to see your point of view. However, I am open to the possibility that both TZDs reduce CV risk, including rosiglitazone based on the results of RECORD (if one takes them at face value). So I see the TIDE trial as both a safety and an efficacy trial, because the goal of glycemic control is to reduce macrovascular complications. I think this is a good question to ask.

The motive for my first comment encouraging the FDA to conduct a 100% audit of RECORD was  to rule out the possibility of any malfeasance. If the audit successfully does that, then the mortality data in RECORD that appear favorable (albeit not significantly) to rosiglitazone might credibly suggest equipoise between the two TZDs. Even though RECORD was an open-label trial, it was also a randomized trial, and therefore it should be given appropriate weight relative to nonrandomized, observational studies such as the CMS study published in JAMA or the meta-analyses that point toward a possible risk with rosiglitazone.

Bottom line, the data regarding CV risk with rosiglitazone are too weak to justify strong arguments. The scientist in me says, “Let’s do more studies to minimize uncertainty.” The clinician in me, however, says, “First, do no harm.” I think I can reconcile both these perspectives!

By the way, all safety trials are not unethical! The FDA has approved many trials designed to rule out unacceptable safety.

Krumholz: That’s very helpful, and a perspective I have not seen: that we are at equipoise with rosiglitazone, and that it actually may be more beneficial than other alternatives. The Bayesian in me feels that the pretest probability is really low — but if you see it as about 50%, then I can see the argument, and it is easier for me to understand the divergence of opinions. But it is hard to believe that RECORD is providing useful information. The problem, of course, is that a full audit would take some time.

By the way, what is an example of a pure safety trial in which you would enroll a loved one who met inclusion criteria?

Kaul: I hope you don’t misunderstand me. I am not trying to say that rosiglitazone might be better than other alternatives. Just that I do not know for sure, consistent with my definition of equipoise.

Before Marciniak (whose work I respect tremendously) presented his analysis, the data from RECORD were viewed as the strongest available evidence against harm with rosiglitazone. Looking at the MACE endpoint, both CV death and stroke data were favorable for rosiglitazone. Although the point estimate for MI went in the opposite direction, the effect on MI was not statistically distinguishable from the effect on CV death or stroke; i.e., no significant heterogeneity of treatment effect was found among the components of the MACE composite endpoint of CV death, MI, or stroke. Data for all-cause mortality (arguably, the endpoint least vulnerable to bias) also appeared to favor rosiglitazone, although the difference was not statistically significant. Marciniak’s analysis has raised enough doubts to question the validity of the RECORD trial results. However, even Marciniak will acknowledge that with only a 12% audit, he cannot prove malfeasance on the part of the trial investigators or sponsors. If  the study results are deemed reliable upon a 100% audit, then we can reasonably assume equipoise between rosiglitazone and alternative treatments (including SU/MET and pioglitazone).

You appear to have prejudged that RECORD does not provide useful information, but that Graham’s CMS study or the Nissen and Wolski meta-analyses do (although both of them suffer from limitations equal to those of the RECORD study). In fairness, both the Nissen and Wolski and FDA meta-analyses (with all their limitations) found increased risk with rosiglitazone compared with placebo, but not compared with SU/MET. By extension, if rosiglitazone is not shown to be harmful compared with SU/MET, and pioglitazone is not associated with risk compared with placebo (ProACTIVE) or with SU/MET (meta-analyses), then why is rosiglitazone or pioglitazone versus add-on placebo in TIDE deemed unethical? Should we rely on an observational study (Graham’s CMS study) to conclude that pioglitazone has been shown to be safer than rosiglitazone and therefore to question the ethical justification for rosiglitazone versus pioglitazone in TIDE?

Don’t even get me started on the “biological plausibility argument” that pioglitazone reduces LDL but rosiglitazone raises it and that this, among other speculated mechanisms, could explain the harm with rosiglitazone. As you well know, Zetia lowers LDL and yet is neutral or harmful with respect to CV outcomes. Similarly, torcetrapib lowered LDL by 25% and yet killed people!

Bayesian estimates should be based on sound information; unfortunately, none exists with respect to rosiglitazone.

Yes, a full audit will take time, but I was told that it could be done. I will say that it should be done!

Vaccine trials (especially those that use inactivated viruses) are classic examples of trials designed to rule out unacceptable safety.

I happen to think that science trumped politics at the Advisory Panel meeting. I continue to hope that it will be allowed to hold sway in the final analysis. I hope you will agree that making sweeping recommendations based on poor evidence does not make good sense. Issues are at stake that go beyond rosiglitazone or the TIDE study. Several decades of progress in evaluating therapies and good methodology are at stake. Currently, no rules of evidence or well-established regulatory standards dictate the quality and the quantity of evidence required to pull drugs from the market. I think it is in everyone’s collective best interest to work towards establishing such standards, driven principally by norms of scientific rigor and integrity. Political and personal agendas have no place in the debate.

3 Responses to “Questions for Sanjay Kaul about TIDE and Avandia”

  1. It is highly problematic when policymakers presume to make value-free decisions. This is what I understand is meant when Dr. Kaul says “Political and personal agendas have no place in the debate”. Guideline groups have long recognized this issue. After evaluating the scientific validity and applicability of the evidence, it is absolutely important to consider and context and values. The need to consider context and values is reflected in the second principle of EBM (the first refers to a hierarchy of evidence affecting the confidence with which one makes decisions). In the case of rosiglitazone, the evidence is such that the inferences that can be derived about the risk:benefit of using this agent vs. using others are going to be weak. Therefore the evidence will not be driving decision making at the policy or clinical levels. What will be driving decisions will be context and values.

    Context in this case indicates that there are other agents available that lower blood sugars, many are cheaper, and most have fewer (or different) side effects (they do not help you gain weight, retain fluids, or increase your risk of bone fractures or macular edema). None can claim to be safer from a cardiovascular standpoint, and none can claim to reduce CV risk (not even metformin). So setting the CV issue aside, there are safer, cheaper, and equally efficacious glucose lowering agents available to patients (and there is always insulin).

    The values and preferences aspect is more complex. Some folks prefer to put on the market many drugs, some of them with limited safety information, to promote and reward innovation, promote competition, and offer patients and clinicians more options. Others prefer to have fewer agents supported by more robust evidence of safety and efficacy. These preferences often conflict (safer drugs may require greater premarketing scrutiny which will increase the cost of development and delay time to market and recovery of investment which may adversely impact innovation, competition, and choice) and people should be explicit about which they value more. I am sure there are other pertinent values to bring to bear on these matters but these two come to mind.

    To pretend that policy making decisions are technical and pay no consideration to context and values is not consistent with EBM, current understanding of policy making, is disingenuous and misleads the public. This is particularly critical when the evidence is nonexistent, it exists but is methodologically weak, or when the tradeoffs are closely balanced. Rather than reject that other considerations are weighing in, I would suggest panels should make the relevant values explicit and members should report their positions on these. Similarly, panels should be explicit about the context for which they are legislating. This increased level of transparency may facilitate debate, offer language for disagreement (is it about the evidence, about the context, or about the values that we are disagreeing? – hint: it is most often that we value things differently), and help explain what appear to be confusing and arbitrary votes. Recognition of the role of values, preferences, and context consideration may also affect panel composition.

    Competing interests pertaining specifically to this post, comment, or both:
    I am a member of the GRADE Working Group that has developed methods to take into account values in the formulation of guidelines. I have written on the philosophy of EBM with Gordon Guyatt and articulated the principles mentioned. My research team works on developing and evaluating strategies to improve the quality of care patients get, by translating evidence into practice and by doing so in a patient-centered way, i.e., in a way that is consistent with patient values and preferences. By our team’s policy (the KER UNIT), we do not take funding from for-profit device and pharmaceutical manufacturers in order to avoid perceived or real conflicts of interest with our work.

  2. Dr. Montori makes compelling arguments in support of the role of context and value judgments in decision making, a stance that is in complete alignment with my own personal practice of not prescribing TZDs given the availability of effective, safer and cheaper alternatives. My lamentation regarding “political and personal agendas” was directed against the intrusion of inappropriate biases into the scientific discourse, not against the exclusion of the important value judgments made by patients and physicians in the course of clinical care.

    Competing interests pertaining specifically to this post, comment, or both:
    None

  3. Great discussion… but wondering how Sanjay makes the defines inappropriate biases… are there appropriate biases? How do we make a distinction? How do we guard against them?