CardioExchange Archive

The CardioExchange Editors interview Jonathan L. Thigpen about his research group’s assessment of the validity of ICD-9 codes in identifying strokes in patients with atrial fibrillation. The article is published in Circulation: Cardiovascular Quality and Outcomes.

CardioExchange Editors: Please describe what you studied and what you found.

Thigpen: We assessed the accuracy of International Classification of Disease, 9th edition (ICD-9) stroke codes in identifying valid stroke events in a cohort of atrial fibrillation (AF) patients. The initial electronic search yielded 1812 events across three stroke centers (Boston Medical Center, Geisinger Health System, and the University of Alabama). All ICD-9–identified stroke events were vetted through manual chart review with final adjudication by a stroke neurologist. AF was verified by electrocardiographic evidence at the stroke admission, 6 months before the admission, or 90 days after the admission.

In addition to assessing the accuracy of the stroke codes alone, we also assessed the combined accuracy of stroke and AF codes, as well as the accuracy of stroke codes in identifying stroke that was associated with AF. These additional steps offer insight about the accuracy and reliability of using only ICD-9 codes to create a “stroke plus AF” cohort. This effort is extremely important given the increasing reliance on ICD-9 codes to identify stroke events and covariates in research, especially research that uses administrative data.

The positive predictive value (PPV) of stroke codes alone was 94.2%. PPVs did not differ across clinical site or by type of event (ischemic vs. intracranial hemorrhage), but they did differ by event-coding position (primary vs. other; 97.2% vs. 83.7%) and by ischemic stroke code (433 vs. 434; 85.2% vs. 94.4%). When combined with validation of AF codes, the PPV of stroke codes dropped to 82.2%. After we excluded ischemic stroke that was caused by a different mechanism (e.g., a vascular procedure, tumor, sepsis), the PPV dropped further to 72.8%. As a separate exercise, manual review confirmed 33 (7.2%) ischemic strokes in 458 events coded as “without infarction.”

Editors: What are the implications for published papers that have used claims data?

Thigpen: Our results indicate that ICD-9 stroke codes alone have limited use in identifying acute strokes in patients with active AF. We suggest that, to limit potential bias, manual verification of stroke is needed to confirm stroke events in the setting of AF.

We recognize that a screening method with a PPV ≥85% (as has been previously suggested) may be adequate for research purposes and is likely to bias estimates very little. However, this thinking rationalizes inaccuracies, which may not be acceptable to some observers. Results derived from screening methods with PPVs <85% are likely to have minimal value. For example, a stroke research study implementing a screening method with a PPV of 80% would mean that 20% of the patients identified as having stroke were in fact false positives, likely leading to significant bias in results.

In any paper that uses ICD-9 screening methods, readers must critically assess the given PPV of the ICD-9 codes and identify the specific screening methods employed (i.e., only including strokes coded in the primary position). The latter is especially important considering that many studies do not report ICD-9 accuracies.

Editors: Do particular studies concern you? Might their results have been different with more-accurate data on outcomes? 

Thigpen: Concerns should be raised about any study that withholds information regarding the accuracy of ICD-9 stroke codes used for case ascertainment. This is especially true if methods to increase accuracies of ICD-9 screening procedures are not implemented (our study reports several available methods, confirmed in previous literature). Without being given the ICD-9 accuracy data for a given study, readers must assume that there are false positives in the cohort, thereby leading to bias (the extent of the false positives and resulting bias may be hard to determine). In recognizing the varying accuracies of ICD-9 stroke codes, we suggest (as many researchers employ) additional screening methods (i.e., manual verification) to increase accuracy.

Increasing the accuracy of ICD-9 stroke codes in the setting of AF will have a variable effect on a study’s results depending on several factors, including (but not limited to) the degree of the increase in accuracy, how the codes were used (to identify patients vs. to ascertain outcomes), and the investigators’ initial conclusions.

Editors: Do you think that ICD-10 will be better?

Thigpen: We know little about the accuracy of ICD-10 stroke codes. Current evidence indicates that the accuracy is similar to that of ICD-9 codes, although ICD-10 codes are thought to be more specific and provide a more intuitive coding method. For example, ICD-10 codes specify the hemorrhage location and source in intracranial hemorrhage, distinguish between thrombotic and embolic ischemic stroke, and include codes for intraoperative and postprocedural strokes. However, we suggest that until ICD-10 stroke codes’ accuracies are further pinpointed and compared with those of ICD-9 stroke codes, manual review of events seems to be warranted.

JOIN THE DISCUSSION

Do Jonathan Thigpen’s findings affect your degree of trust in how well ICD-9 codes identify strokes in patients with atrial fibrillation?