May 14th, 2014
BMJ Articles Critical of Statins Provoke Kerfuffle
Larry Husten, PHD
The authors of two BMJ articles have withdrawn statements about the adverse effects of statins. The papers inaccurately cite an earlier publication and therefore may overstate the incidence of adverse effects. As a result, the two papers have drawn much criticism and set off a kerfuffle involving the editor of BMJ and a highly prominent British trialist who is demanding a full retraction of the articles. But the controversy probably won’t be resolved any time soon, since an independent panel, which is being assembled to decide the issue, has not yet begun its work.
In an editorial published in BMJ, the journal’s editor-in-chief, Fiona Godlee, explains the reason for the corrections and the lingering controversy. In October 2013, BMJ published two articles that cited the same study by Zhang and colleagues to support the statement that statin side effects occur in 18-20% of patients. The first article, by Abramson and colleagues, reanalyzed data from the Cholesterol Treatment Trialists’ (CTT) Collaboration. The second article, by Aseem Malhotra, questioned the role of saturated fat in heart disease. But, writes Godlee, the articles “did not reflect necessary caveats and did not take sufficient account of the uncontrolled nature” of the data in the paper by Zhang and colleagues.
The aim of the editorial, she writes, is “to alert readers, the media, and the public to the withdrawal of these statements so that patients who could benefit from statins are not wrongly deterred from starting or continuing treatment because of exaggerated concerns over side effects.”
Following the initial publication of the two BMJ articles, the head of the CTT Collaboration, Rory Collins, contacted Godlee on several occasions to express his concerns about the papers, though he declined requests to respond in BMJ. The BMJ editors agreed with the authors of the two studies on the wording of a published correction (see below), but Collins still requested a full retraction. Godlee reports that she is uncertain “whether the error is sufficient for retraction, given that the incorrect statements were in each case secondary to the article’s primary focus.” As a result, BMJ is convening an outside panel of experts “with no dog in this fight.”
Here is the full wording of the two corrections:
Should people at low risk of cardiovascular disease take a statin?
The conclusion and summary box of this Analysis article by Abramson and colleagues (BMJ 2013;347:f6123, doi:10.1136/bmj.f6123) stated that side effects of statins occur in about 18-20% of patients. The authors withdraw this statement. Although it was based on statements in the referenced observational study by Zhang and colleagues, that “the rate of reported statin-related events to statins was nearly 18%,”(1) the article did not reflect necessary caveats and did not take sufficient account of the uncontrolled nature of the study.
Zhang et al observed that the rate of statin related events found in their study (18%) was “substantially higher than the 5% to 10% usually described in randomized, placebo-controlled, clinical trials.” Two caveats must be considered. As Zhang et al point out, the rate of statin related events reported in their study was uncontrolled and therefore may be inflated because events attributed to statins might have occurred in a placebo group as well. In addition, although Zhang et al do not make this point, the 5-10% rate quoted by Zhang et al as having been observed in randomised trials was, in many cases, similar in both active and placebo groups.
The exact rate of statin related adverse events in people at low risk of cardiovascular disease remains uncertain. Clinical trials may underestimate the frequency of statin related adverse events because of patient selection, exclusion of older patients and those with comorbid conditions or potential drug interactions, under-representation of women, and selection bias created by willingness to participate in a clinical trial. In addition, when compared with the full clinical study reports, published accounts of clinical trials in medical journals report only a minority of adverse events.(2) Access to the full data from the trials of statins would help to determine the comparative rates of serious adverse events in statin and control groups but probably would not help to determine the frequency of less than serious adverse events.
The authors also mistakenly reported that Zhang et al found that “18% of statin treated patients had discontinued therapy (at least temporarily) because of statin related events.” The correct interpretation of the data, as confirmed to The BMJ by Zhang et al, is as follows. Based on review of structured electronic medical record categories and automated review of unstructured narratives from follow-up visits of 107,835 patients over eight years, 18,778 of all study patients (17.4%) had a statin related event documented during the study. Among those who experienced a statin related event, only 59.2% had statin therapy discontinued at least temporarily. However, because of possible miscategorisation resulting from the limited options in the electronic medical record for recording reasons for discontinuation of statin therapy, Zhang et al concluded that “as many as 87%” of these discontinuations could have been due to statin-related events. This equates to up to 9% of the study population having possibly discontinued statin therapy as a consequence of statin related events, rather than the 18% cited.
The primary finding of Abramson and colleague’s article—that the Cholesterol Treatment Trialists’ data failed to show that statins reduced the overall risk of mortality among people with <20% risk of cardiovascular disease over the next 10 years—was not challenged in the process of communication about this correction.
1 Zhang H, Plutzky J, Skentzos S, Morrison F, Mar P, Shubina M, et al. Discontinuation of statins in routine care settings. Ann Intern Med 2013;158:526-34.
2 Wieseler B, Wolfram N, McGauran N, Kerekes MF, Vervolgyi V, et al. Completeness of reporting of patient-relevant clinical trial outcomes: comparison of unpublished clinical study reports with publicly available data. PLoS Med 2013;10:e1001526 .
Cite this as: BMJ 2014;348:g3329
Saturated fat is not the major issue
This Observations article (BMJ 2013;347:f6340, doi:10.1136/bmj.f6340) by Aseem Malhotra stated that a recent “real world” study of 150<thin>000 patients who were taking statins showed “unacceptable” side effects—including myalgia, gastrointestinal upset, sleep and memory disturbance, and erectile dysfunction—in 20% of participants. The author withdraws this statement. Although it was based on statements in the referenced observational study by Zhang and colleagues that “the rate of reported statin-related events to statins was nearly 18%,” (1) the article did not reflect necessary caveats and did not take sufficient account of the uncontrolled nature of the study.
1 Zhang H, Plutzky J, Skentzos S, Morrison F, Mar P, Shubina M, et al. Discontinuation of statins in routine care settings. Ann Intern Med 2013;158:526-34.
Cite this as: BMJ 2014;348:g3332
The slips by Abramson et al, repeated by Malhotra, are easy to make. They must be, because they are made so commonly in the lay media and lay blogs.
A. Mistaking lack of change in mortality in low risk patients during trial duration as evidence that treatment is ineffective.
The decision to embark on primary prevention is intended to be not for a few months or a few years but for life. The earlier one starts (i.e. at lower short-term risk) the longer one should be followed if the trial is to directly observe the lifespan gain. Because lengthy trials running into many decades are prohibitively expensive, trials addressing low risk populations usually use non-mortality primary endpoints which are more numerous and therefore have greater chance of showing the consequences of cardiovascular event reduction.
B. Forgetting why we do randomized placebo controlled trials, and therefore ignoring them
People are highly susceptible to feeling better or worse with intervention. When we choose to have an intervention, we are biased to believe it was effective. (How many of us think that the university or medical school we chose was anything other than very good?) When we have an intervention imposed on us, we are biased to believe it was harmful. (How many of us think the administrative forms imposed on us by our employers are anything other than a waste of time?)
For trials of symptomatic remedies, that patients would voluntarily choose, the placebo arm is there to stop us falsely believing that the remedy is improving symptoms.
For trials of interventions imposed on patients who feel healthy, the placebo arm is there to stop us falsely believing that the intervention is causing symptomatic harm.
The 2 papers in the BMJ forgot this. They not only forgot to subtract the rates of symptoms in the placebo arm, but picked on data which had no placebo arm at all. Placebo-subtracted data are available at http://tinyurl.com/Side-effects-CAUSED-by-statins
Abramson et al, and Malhotra, are certainly entitled to voice their opinion that statins are not worthwhile for many people. They may be right, since what people consider worthwhile varies much more widely than guideline bodies assume (http://blogs.jwatch.org/cardioexchange/voices/a-new-tool-to-discuss-primary-prevention-with-a-statin-with-patients-life-expectancy-gain). However if there is an important scientific error in an article this can be corrected calmly and clearly.
The BMJ Editor has been surprisingly forthright and scientifically correct in her media appearances on this in the last few days, which one does not always see from Journal Editors. She stated that the Collins argument of equal side effects in active and placebo arms of trials was correct, and that for that reason she was issuing the corrections. She went on to add the warning that if the trial data had been systematically manipulated before publication, however, then any inferences from them may be unsound.
This escapade should remind us all that we all make mistakes. The only way to never make mistakes is to never say anything worthwhile, instead limiting oneself to recycling meaningless platitudes, a habit of far too many of our elite. We should congratulate the BMJ for being transparent in a scientific matter that affects the welfare of millions.