CardioExchange Archive

For more than a decade, cardiac stem cell therapy has attracted an enormous amount of attention, promise, and research dollars. Now an original and important new study published in BMJ finds that many of the most promising results in the field are illusory and that the potential benefits of stem cells to treat heart disease are probably far more modest than we’ve been led to believe. The study also raises disturbing questions about ethics and research conduct (and misconduct) in a high-flying field.

Researchers in the U.K., led by Darrel Francis, closely scrutinized 133 reports from 49 different clinical trials testing autologous bone marrow stem cells in patients with heart disease. They found an astonishingly large number of discrepancies in the reports — altogether more than 600 discrepancies, ranging from minor oversights to serious unexplained errors and apparent deceptions. Many errors were mathematical or statistical errors, while others were more general, such as conflicting descriptions of studies as either a prospective randomized trial or a retrospective observational study.

The key finding of the BMJ study is that there was a strong correlation between the number of discrepancies in a study and the reported improvement in heart function as measured by left ventricular ejection fraction (LVEF). The five trials with no discrepancies reported no improvement in LVEF (-0.4%). In stark contrast, the five trials with the highest number of discrepancies — each with more than 30 discrepancies — reported a very large and, if true, clinically significant improvement in LVEF (+7.7%). This effect was consistent: the more errors there were in the study, the more likely the study reported a large treatment effect.

The authors summarized their finding:

Our study shows that scientists who achieve progressively better consistency of reporting find progressively smaller effects on ejection fraction of treatment with stem cells derived from bone marrow. In trials with a discrepancy count of zero, the ejection fraction effect seems to be zero.

The authors said they were unable to obtain explanations for the discrepancies: “We do not know the cause of the discrepancies. We have asked for resolution of over 150 discrepancies through journals. None were resolved, although we found it triggered correspondence from lawyers.”

By sheer coincidence, the BMJ publication occurs simultaneously with the publication of a Cochrane review of stem cells for heart disease. The two papers don’t completely overlap, but they are in many ways congruent. Analyzing the literature, the Cochrane reviewers found “some evidence that stem cell treatment may be of benefit.” But, they noted, “the quality of the evidence is relatively low because there were few deaths and hospital readmissions in the studies, and individual study results varied. Further research involving a large number of participants is required to confirm these results.”

Similarly, the BMJ authors note that “viewing all the studies together as a single entity, there is on average a positive effect.” But, they warn, “averaging effect size across all studies might therefore not be wise because it does not reflect their varying factual accuracy.” Although Cochrane is known for its rigorous methodology, it is generally not within the purview of Cochrane reviewers to seek out errors and inconsistencies in individual studies.