CardioExchange Archive

In a recent Viewpoint in JACC: Cardiovascular Interventions, Marso and colleagues expressed grave reservations about the application of appropriate use criteria for PCI in a controversial study published last year in JAMA by Chan and colleagues, which found that only half of PCIs performed for nonacute indications were classified as appropriate. Interventional cardiology editors Rick Lange and David Hillis asked CardioExchange members for their opinions on this topic. In this series of posts, Lange and Hillis question the main protagonists in the debate, Steven Marso and Aaron Grantham, on the one side, and Paul Chan and John Spertus on the other side. All the authors are affiliated with the Saint Luke’s Mid America Heart Institute in Kansas City, Missouri. Click for Part 1 and Part 2 of this debate.

Lange & Hillis: A recently published study that garnered national attention relied on data collected from the National Cardiovascular Disease Registry (NCDR) to assess the appropriateness of PCI in a large contemporary United States patient cohort. Would you comment on the reliability of these data? 

Chan & Spertus: Extensive efforts were undertaken to assure the accuracy of the mapping and there is extensive site training provided to NCDR data collectors to insure the best possible abstraction of data. An internal audit among 11 sites also found exceedingly high accuracy rates (>95%) in coding for their 127 inappropriate cases, and none of the divergent cases suggested that reforms were needed prior to publishing the data (e.g. one site mentioned that ‘they do not believe their stress test results’, which is hardly a compelling reason to change the AUC or its reliance on the magnitude of ischemia as defining patients more appropriate for revascularization). We do recognize that there may be difficulties in the data collection due to a) poor documentation in the medical record surrounding patient characteristics and the indications of the procedure (including symptom status); b) non-uniform reporting of stress testing that warrants improvement in the accuracy of interpretation and reporting; c) variability in the interpretability of angiographic stenosis; etc. However, we believe that the AUC are serving as an important stimulus to improve all of these aspects and will lead to continued improvements in the accuracy and interpretability of the AUC data.

Marso & Grantham: The NCDR is an important vehicle for research, quality improvement, and performance feedback in the United States. Like any registry it has strengths and weaknesses. This registry is voluntary, has limited auditing and no ongoing adjudication. These pragmatic design features enable broad participation, the findings from ongoing quality improvement projects are perhaps even more generalizeable. Furthermore, this approach helps to restrain the cost of maintaining this registry. We suspect the collection of routine data elements including age, sex, stent type and binary covariates such as death are likely recorded reliably.

However, it is problematic to distill complex information into discreet data points for use in the NCDR Registry. The case example for this is the need to assess future risk based upon stress test results. Is the test normal or abnormal and if abnormal low, intermediate or high risk? Unfortunately stress test result reports are not standardized and many do not comment specifically upon the risk of the result. In order to categorize a PCI as appropriate, uncertain or inappropriate it is required that data abstractors often “interpret the interpretation”. This level of data collection is usually performed by highly trained personnel in core laboratories in high cost, randomized controlled clinical trials. This data collection requirement is problematic for a number of reasons. Firstly, the quality of stress testing in the United States is highly variable. This is perhaps an insurmountable problem. Secondly the AUC’s guidance on whether a stress test is high risk or not is far too vague. Thirdly, there is no systematic method in which clinicians summarize the interpretation of the stress test. This leads to substantial challenges when attempting to record this information. Finally there is no requirement for training in order to enter data into a case report form. At some institution it is done by first year fellows while others hire professional abstractors or require interventional cardiologists to enter the data. We believe this leads to substantial variability in categorizing a PCI as A, U or I. This is the crux of the problem with the JAMA paper. It is just as likely that the inappropriate (I) rate of 11.6% among stable patients and the variability found between hospitals is related to variable quality in these interpretations as it is to do with case selection.