January 15th, 2010
Evaluations
Andrew M. Kates, MD
This Journal Watch summary of a recent article from the New England Journal of Medicine made me consider further an issue that seems to be all-consuming: evaluations. In training programs across the country, we are required to give and receive evaluations on a regular basis. The goal, at least in part, seems straightforward enough — trainees require feedback to effectively change (or maintain) certain behaviors.
In many programs, these evaluations are done in the “360” style; that is, fellows evaluate fellows, nurses evaluate fellows, patients evaluate fellows…well, you get the idea. The benefits of these circular evaluations is apparent –- feedback early in the course of training may be one of the most effect means for reinforcing good behaviors and remodeling bad ones. The potential drawback is also apparent, although much less discussed and, in reality, quite rare; we all have nightmares about a colleague or patient with an axe to grind making something up or blowing something out of proportion in order to get us in trouble.
What do you think of these evaluations? Is constructive criticism from your colleagues helpful or possibly hurtful? Do you find evaluations of your attendings a beneficial practice? Are you at all afraid to be honest in evaluations of peers because you want to avoid confrontations and avoid attending evaluations so as to avoid possible repercussions?
5 Responses to “Evaluations”
NEJM Journal Watch — Recent Cardiology Articles- Anticoagulation in Patients with Cancer-Associated Pulmonary Embolism: How Long is Long Enough?
- Spironolactone Use After Percutaneous Intervention for Acute Myocardial Infarction
- CRISPR/Cas9 Gene Editing for Transthyretin Cardiomyopathy
- Does Colchicine Reduce Cardiac Events After Myocardial Infarction?
- First-Line Therapy for Ventricular Tachycardia After MI: What's the Best Approach?
The Challenge
The problem, it seems to me, is that honest feedback is difficult in the close quarters of a fellowship. In particular, Fellows will be reluctant to share their views with people who may be in a position to influence their future career opportunities. Fellows are loquacious off the record — but on the record it is difficult to get any candor — and understandably so. Yet, the insights from Fellows are valuable. I wonder if we should have quarterly Fellow interviews conducted by an impartial outsider who can distill them for distribution to the faculty. There must be some way to get this information in a way that protects those who are vulnerable.
Inherent Fallacies
With a very small number of fellows in most programs, anonymity is difficult to maintain. Secondly, as more and more fellows strive for sub-specialty training, ‘honesty may not be the best policy’ because even constructive criticism is after all criticism for people who could potentially support your carrer aspirations. I feel, the 360 degree evaluation beats some of its purposes, since candor is quite difficult to expect and achieve in a very competitive enviornment. Political correctness will remain the order of the day. A neutral external evaluator, with distilled quaterly input and feedback, may partly help the purpose.
Having said that, it is well recognized that the current system of evauation does help programs to identify ‘outliers’ at both ends of the bell curve; but whether this system will help the ‘judged’ to change behaviors, who already are at such an advanced level of their careers with a personality of their own, will be extremely difficult, if not impossible, to achieve.
Constructive criticism is a Good Thing
I agree that candid evaluations from fellows are sometimes hard to come by, but I think that it is equally a problem of faculty not being willing to criticize fellows. Most people (fellows and faculty) don’t want to “rock the boat”, but that could lead to people slipping through the cracks instead of being held accountable. At the end of the day, the people that you really want as your colleagues are the ones who are constantly self-assessing and who want to improve themselves, even if it means hearing some criticism. Pooling anonymous criticism helps, which is the way that our faculty receive evaluations. That is, until there are 5 (maybe 4?) evaluations, they don’t get any of them, then they get all 5 at once, and they are all anonymous. Fellows have a shorter time-frame to work with, so their evaluations are direct from the attendings and are therefore not anonymous (which is why many attendings “hold back”). I’d also be interested to hear some comment about actually ^dealing with^ the negative comments that do come up. Dealing with troublesome faculty who repeatedly get poor evaluations is probably even harder than dealing with fellows in the same situation!
Our strategies
As a fellowship director I certainly agree with the lack of candor in the evaluation process, and have on several occasions had to explain to trainees that the written record only reflects a small portion of what the faculty communicates about performance. I have tried to do a few things to improve the utility of the evaluation process. I get copied on all the evaluations, so I provide intermittent feedback to faculty members about the quality and appropropriateness of evaluations, praising those who use the full range of scores available, and providing immediate and constructive feedback to the softies who grade average as 8 and axe murderers as 7 on the 9 point scale!
We also do an in depth “biopsy” of the program now twice per year, using a detailed survey monkey developed by my co-director, Amit Khera. This is anonymous and the fellows have certainly been much more open about problem areas and problem faculty in these surveys than in the standard end of month survey.
This is a challenging process. Getting fellows and faculty to fill out evaluations is tough enough and getting honest feedback tougher. Dealing with the substandard evaluations can be difficult as well. Along that line, Chris brings up a good point – how do we (as faculty) respond to substandard evaluations of attendings? While it is (somewhat) easy to provide criticism (both constructive and otherwise) to fellows, it may be more challening to provide feedback to our colleagues. I am interested to here from both fellows and fellowship directors what their experience has been. For fellows, have you had a positive experience in changing behavior of faculty? For fellowship directors, have you been able to change behavior of faculty, have you met resistance or have you been reluctant to do so approach collegues?