November 18th, 2013
Controversy Erupts Over Accuracy of Cardiovascular Risk Calculator for Guidelines
In the face of a highly critical story in the New York Times by Gina Kolata about the new cardiovascular guidelines, authors of the guidelines and leaders of the American Heart Association (AHA) and the American College of Cardiology (ACC) defended the value and integrity of the guidelines.
The Times story claims that the cardiovascular risk calculator used to assess individual risk in the new guidelines is deeply flawed: “In a major embarrassment to the health groups, the calculator appears to greatly overestimate risk, so much so that it could mistakenly suggest that millions more people are candidates for statin drugs.” The story quotes former ACC president Steve Nissen: “It’s stunning. We need a pause to further evaluate this approach before it is implemented on a widespread basis.”
But the guideline authors and AHA/ACC officials strenuously defended the guideline at a news conference Monday morning at the AHA meeting in Dallas. They said the new risk calculator is far superior to previous efforts, incorporating far more data that now includes stroke assessment and, for the first time, provides specific predictions for African Americans.
The assembled officials rejected any call to delay implantation of the guidelines, but one author, Donald Lloyd-Jones, said that “over time we will modify the risk scores so that they get better and better.”
“We think we’ve done our due diligence,” said AHA president Mariell Jessup. “We have faith and trust in the people who developed the guidelines.”
The Times story is based on a commentary scheduled for publication in the Lancet on Tuesday that is highly critical of the new calculator. The paper, written by Harvard Medical School’s Paul Ridker and Marcia Cook, states that the calculator overpredicts risk by 75% to 150%. At a hastily assembled meeting on Saturday night at the AHA meeting in Dallas, Ridker presented his findings to leaders of the AHA and ACC.
One source of confusion is that Ridker and Cook had sent their criticism to the National Institutes of Health’s National Heart, Lung, and Blood Institute (NHLBI) a year ago, but, according to Kolata, their concerns were never passed on to the AHA and the ACC. At the Saturday night meeting Ridker presented data showing that the risk calculator calculated much higher risk for the population of people who had been followed in three large studies: the Women’s Health Study, the Women’s Health Initiative, and the Physician’s Health Study. Kolata writes:
“…the calculator overpredicted risk by 75 to 150 percent, depending on the population. A man whose risk was 4 percent, for example, might show up as having an 8 percent risk. With a 4 percent risk, he would not warrant treatment — the guidelines that say treatment is advised for those with at least a 7.5 percent risk and that treatment can be considered for those whose risk is 5 percent.”
In a draft of the Lancet article I obtained, Ridker and Cook write:
“…it is possible that as many as 40 to 50 percent of the 33 million middle-aged Americans targeted by the new ACC/AHA guidelines for statin therapy do not actually have risk thresholds exceeding the 7.5 percent level suggested for treatment. Miscalibration to this extent should be reconciled and addressed in additional external validation cohorts before these new prediction models are widely implemented.”
But, according to the guideline authors, Ridker’s position is not new and is discussed in the guidelines. The people included in the three trials presented by Ridker are unusually healthy and are “not really representative of the broad risk,” said Lloyd-Jones. “So it doesn’t surprise me at all that our equations overestimate risk in these groups.” He also said that he thought the overestimation of risk was largely because patients with the highest risk were already taking risk-reducing statins.
AHA president Mariell Jessup said that she wished “there had been this much attention in the past when we used prior risk calculators. If you were an African American and had a stroke” the guidelines didn’t help.
Lipid panel co-chair Neil Stone said that 7.5% is not “an absolute cutoff. This is not the end of the discussion, it’s the start of the discussion.”