June 7th, 2013
“Your Mortality Risk Is 11.827%”
John E Brush, MD
An 85-year-old woman named Betty presents with a non–ST-segment elevation myocardial infarction. Despite her advanced age, she is active, lives by herself, and continues to work as a volunteer in the hospital’s gift shop. She has hypertension, well-controlled diabetes, and a serum creatinine level of 1.8 mg/dL. She also has recurrent angina despite intensive medical therapy, so cardiac catheterization is performed. It reveals a high-grade stenosis of the left-main coronary artery, as well as complex multivessel disease. The Society of Thoracic Surgeons’ online risk calculator estimates Betty’s mortality risk after urgent CABG to be 11.827%.
What does that number mean? And how do you explain it to Betty so that she knows what to expect? Probability is a way to quantify uncertainty about the future. But what exactly does an event’s numeric probability tell us?
What Probability Means
Mathematicians and philosophers have been arguing about the meaning of probability for about 300 years. They have settled on two distinct notions: a frequentist notion and a personal notion. The frequentist dogmatists, such as R.A. Fisher, say that probability is a count of events — how things turn out over the long run, like the number of adverse events documented in an observational study. The personal probability dogmatists, such as Bruno de Finetti, say that probability represents a degree of belief or conviction — a forward-looking way to predict single events, like the chance that it will rain tomorrow or that Betty will make it through a risky operation.
As physicians, we use both notions of probability, even though we don’t give them much thought. We can look back at a database to find the frequency of survival in patients just like Betty. And we can switch to a personal notion of probability to talk with Betty about her chances. Tell Betty she has an 11.827% risk for mortality, and she may laugh at the false precision and the illusion of certainty. She might say, “What do you mean, doc? For me, it’s either 0% or 100%.” We might respond, “If I had 10 patients like you, 9 would survive surgery.” What we mean, of course, is that we are about 90% sure that Betty will make it through the operation. But that’s a tough idea to grasp when you’re the one going under the knife.
Probability’s Many Faces
Probability comes in various forms: simple, compound, conditional, complementary, and cumulative. And there are “odds” — namely, a probability divided by its complementary probability. If there is a 75% probability of rain tomorrow, the odds of rain are 0.75 divided by 0.25, or 3 to 1. Odds are a handy format, because prior odds can be multiplied by other numbers such as likelihood ratios or Bayes factors, to give posterior odds. Using odds, we can calculate probability estimates in our heads or with a calculator. For example, if we think a patient has a 50% prior probability of coronary artery disease, the odds of CAD would be 0.5/0.5, or 1. If an imaging stress test is positive, we multiply the prior odds (1) by the positive likelihood ratio for an imaging stress test (6); that yields a posterior odds value of 6. Converting odds back to probability (p=odds/odds+1) gives us a posterior probability of CAD of 86%.
We often use subjective quantifiers to express probability and make intuitive judgments. We throw around terms such as “unlikely,” “possible,” “probable,” and “almost certain” to express probability. Psychologists tell us that we frequently make mistakes when we subjectively estimate probability. We could turn the whole thing over to computers, but most patients don’t want cold calculations. So it seems important to calibrate our intuition from time to time using hard numbers. But first we have to wrap our minds around the meaning of probability.
We speak about probability all the time in medicine, yet we rarely stop to reflect on it. Philosophers such as Ian Hacking have written extensively about the meaning of probability. I have a chapter on it in my iBook, The Science of the Art of Medicine (free and available for download through the iBookstore). In my book, I discuss the history of probability ideas, how probability is viewed and calculated, and the various mistakes we sometimes make when we try to estimate probability.
What is your concept of probability, and how do you use it in clinical practice? How can we refine our notions of probability to improve medical decision making? I invite you start the conversation with me and other CardioExchange members right here.
10 Responses to ““Your Mortality Risk Is 11.827%””
NEJM Journal Watch — Recent Cardiology Articles- Sex-Based Disparity in CABG Outcomes: Are We Moving Toward Parity?
- Can Genetic and Social Risk Scores Improve Cardiac Risk Prediction?
- Extensive Ablation vs. Pulmonary Vein Isolation in Persistent Atrial Fibrillation
- Can a Novel Stent Design Improve Long-Term Outcomes of Percutaneous Coronary Intervention?
- Are Angiographically-Derived Coronary Physiology Measurements Ready for Use in the Cath Lab?
Patients’ relatives often quiz us on probabilities here in Greece. Some approaches I use:
“Well, things are very serious, the probability of surviving septic shock is only about 50%, about the same as surviving bubonic plague.”
“The surgery is considered high-risk, the probability of not surviving is about 10%, but on the other hand not performing surgery on your relative would equal a slow and painful downhill to death, and the chances of survival are quite high. Mind you, the road could be long and rocky, with many complications in the meantime, because simply surviving is not our goal. All in all,it’s still worth taking the risk and I would recommend it if the patient was my mother.”
“There is no such thing as no risk, but it is safer than travelling on the highway.”
Recently I’ve been using an App (Qx Calculate) that has many “calculators” and one that has some seeming merit is the (Revised) Lee Criteria for preoperative assessment for the risk of MI, PE, VF, cardiac arrest, or heart block. If I plug in the numbers for a very common patient I get consulted on: a low risk surgery in a patient with diabetes and a prior MI, the score reports a “moderate risk”, specifically 6.6%. In my anecdotal experience this is a remarkably high number compared to what I have seen in > 20 years of practice. My point is, I have a hard time reconciling what the literature tells me, and what my experience has been with this particular patient. I have been reluctant to report this type of information to my referring doctor, the patient and their family. I usually say that the patient is an “acceptable” risk, and tell the patient and family that there are no certainties in medicine. Perhaps a re-evaluation of what low, medium, and high risk means to physicians and patients is in order. (In the patient you describe above the calculator computes high risk, >11%).
I too am looking forward to seeing how others approach this issue.
I always have this question. We use statistics to calculate risk, probabilities, odds, etc. Then we develop a program to calculate this and use the information to take decisions. This information change our behavior. If we change our behavior… Do we change the probabilities an so on…?
“What we mean, of course, is that we are about 90% sure that Betty will make it through the operation.”
To be on the frequentist side, do we even know what we mean when we say that, we are about 90% sure? When regression methods are used to derive point-estimates of risk for individuals, both first order errors (i.e., estimate variance due to the regression process) and second order errors (i.e., estimate variance due to variance in the covariables) contribute to lack of precision. So when we calculate a risk score, and say the mortality risk is 10%, the 95% confidence interval for that estimate could be 9%-11%, or it could be 4-22%. Unfortunately the risk calculators never provide that confidence interval; nor is it clear how we would use that information if they do.
The epidemiologists will also tell us, as Dr. Payne notes, that the risk calculators are calibrated for the population in which they were derived. For instance, the commonly used STS score is known to over-estimate risk in contemporary cohorts of cardiac surgery patients. So when faced with patients in front of us, are we certain that they are representative of the population in which the risk score was derived?
I do believe that at a population level, evidence-based decision making through quantitative risk assessment provides overall benefit by correcting heuristic biases. But I think in addition to reflecting on the meaning of probability (is Schrodinger’s cat alive or dead?), we should also develop more evidence for how to communicate risk, and how to tailor what is communicated to individual patients. Does communicating risk as probability or qualitatively work better for certain patients rather than others? How do we measure that? If a complication happens, how is patient perception of the complication influenced by the way in which its risk was described? We often intuit answers to these questions and call it the art of medicine, but we ought to also look for their answers empirically.
I find it more informative to inform patients about their chance to being alive with or withour treatment. For instance, using the data from the recent Cochrane meta-analysis of statin treatment in primary prevention, you can tell the patient that his chance of being alive during the next three years is 94.8 % without treatment, but if he take a statin drug every day he can improve his chance to 95.6 %
It’s not just about probability. It’s about how we phrase it. A 5% chance of dying and a 95% chance of surviving are perceived differently. We are guilty of mixing relative and absolute risks. With anticoagulation for AF how often to people say “warfarin reduces your risk of stroke by 2/3rds and there is only a 2% risk of serious bleeding each year”? It also seems to be about how we are built to deal with numbers. We are poor at grasping small figures. Probability is perceived in a non-linear fashion too. The difference between 0% and 1% is much larger than 1% to 2% etc. Behavioural economists such as Kahneman and Tversky have written extensively on this topic.
I help develop decision aids for patients and would like to suggest a couple summaries of the research on risk communication issues mentioned in the thread, such as which risk format to use and whether to frame risks in terms of gain (survival), loss (mortality), or both.
* Helping Patients Decide: Ten Steps to Better Risk Communication (2011 JNCI)
http://jnci.oxfordjournals.org/content/early/2011/09/19/jnci.djr318.full
* Communicating risk (2012 BMJ)
http://www.bmj.com/content/344/bmj.e3996
* Using alternative statistical formats for presenting risks and risk reductions (2011 Cochrane)
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD006776.pub2/abstract
I think it’s very much an evolving evidence base, with heterogeneous study designs and some inconsistent findings. There’s a lot still to tease out about how the context and details in the risk presentations influence risk perception and decision-making. The field is growing, and I agree with the comments that there are many insights to be gleaned from disciplines outside medicine, too. Also agree with the point that some of the biggest challenges come from uncertainty and lack of generalizability of underlying data.
People have raised a number of very interesting issues.
As Dr Ye noted, the STS risk calculator doesn’t give 95% confidence intervals around the point estimate of risk. In fact, the calculator gives the point estimate calculated to 3 decimal places, implying an extraordinary degree of precision. I suspect that there are very few 85 year old women in the population from which the risk model was derived, and the 95% confidence intervals around the point estimate for this patient are probably pretty wide.
The estimate doesn’t inform us about the risk of the alternative – medical therapy. To make a good therapeutic decision, it would be helpful to know the risk of alternative treatment strategies.
Dr. Dayer noted that when we discuss probability with patients, we can frame our discussion in various ways, which might change the patient’s perception of risk. Should we say the mortality risk is 12%, or should we say that the survival rate is 88%?
He noted that we do a poor job estimating probability and tend to exaggerate the probabilities at the extremes. Psychologists talk about “the possibility effect” at the low end and “the illusion of certainty” at the high end.
Mr Parmet talked about decision aids and statistical formats. Patients generally don’t understand percentages. It’s better to use natural frequencies. Also, we can use pictograms to visually express rates and make the probabily ideas more visual for patients. He provided links to some very useful resouces.
Thanks to all for a very robust discussion about probability and its ramifications. Shouldn’t we explicitly teach probability ideas to our students, residents and fellows? How do you teach these important concepts to your trainees?
I give the example of framing the decision for primary prevention of sudden death with an ICD. assume an annual decrease in death from 7 to 5%.
(1) If you don’t get one, your risk of death will climb by 40%. (2) The ICD will decrease your risk of death by 30%. (3) The risk of death in 5 years will decrease from about 35 to 25%. (4) The ICD will decrease your risk of death from 7 to 5% per year. (5) An ICD will save 1 out of 10 implanted patients over five years.
I believe the statements are roughly equivalent, but in decreasingly convincing order. Not many patients would elect an ICD if given proposition (4), particularly if combined with the side-effects of ICD’s.
Indeed, our decision framing can make a mockery out of informed consent. The referenced tools and education can help. But….is there anything “wrong” with any of the above statements? Should they all be presented? Is there an ethical issue here? I think (1) and (2) are ethically problematic, as they omit absolute risk.
I have posed this scenario to students. They think about it….but I’m not sure how it affects them.
Terrific example. This is a great demonstration of how relative risk reduction can exaggerate a treatment effect. Absolute risk reduction is a more accurate, but I like the reciprocoal, number needed to treat, even better.
With NNT, you can boil the whole thing down to a simple declarative sentence: “You need to treat 10 patients with an AICD to save one life over a period of 5 years.”
To use ARR and NNT, however, you need a comparison with a control group from a randomized controlled trial. For Betty, we don’t have that. We just have a number that may not make much sense unless we can put it into context and explain it properly.