July 23rd, 2012
A Proposal To Improve The Value Of Observational Studies
Harlan M. Krumholz, MD, SM
I believe that observational studies can reveal important truths and have a critical place in the portfolio of clinical research. However, I sometimes wonder, when I see a study, just how it was conducted. Was the study question clearly defined before the analyses were begun…or did the study question emerge only after the investigator conducted many undirected analyses? In an Editor’s Perspective that I wrote, I question whether it would be useful to have journal’s require authors to disclose the methods history. Should they post their original study protocol in an online appendix? If the study was exploratory, should they disclose that intent? Would that help us better interpret the findings? I would be interested in your thoughts. I am Editor in Chief of Circulation: Cardiovascular Quality and Outcomes and we are discussing whether we should require information about the methods history for each observational study that is submitted. We would benefit from hearing your thoughts. What should we do?
If you are interested in my Editor’s Perspective, please see
5 Responses to “A Proposal To Improve The Value Of Observational Studies”
NEJM Journal Watch — Recent Cardiology Articles- Persistent Symptoms After Valve Replacement for Severe Aortic Stenosis
- Is Diltiazem Safe in Patients with AF Taking Apixaban or Rivaroxaban?
- Does Lowering High Childhood Lipid Levels Prevent Adverse Cardiovascular Events in Adulthood?
- Self-Expanding Supraannular Valve vs. Balloon-Expandable Valve in TAVR for Small Aortic Annulus
- A New TAVR Prosthesis for Aortic Regurgitation Shows Promise
Yes Yes and Yes
Yes
Harlan,
Any enhancement of the clarity and detail of the Methods of such studies should be welcomed. My two major issues with studies of this type is lack of detail in the methods and over-interpretation of the findings. My suspicion is that much of this is driven by a desire to emphasize the novelty (or avoid anything that might be seen to compromise the novelty) of the findings, for novelty sells (journals and products).
Whilst I’m at it, I’d add that journals that relegate the Methods section to the back of the paper not only disrupt the flow of information but are sending a subtle message about the relative importance of understanding how the study was done. Another enhancement worth considering is that of pre-specified analysis plans which are used, albeit mainly in large prospective studies.
Interesting proposal, Harlan. I am generally in favor of the concept so that people can more appropriately interpret the published observational literature. However, I do think that keeping it relatively simple is probably an advantage. For a true “exploratory” study (like so many), I suspect it would be completely impossible to lay out the full evolution of the methods.
So, my suggestion would be to stick to disclosure of whether the final analytic protocol (at least the primary analysis) was pre-specified or whether any component of the primary analysis was exploratory. I sincerely doubt that anyone would be able to intepret the more subtle shades of gray that exist between these 2 extremes.
I’m not certain I agree that this should be necessary, but it would certainly be interesting. And I agree with Dr Cohen’s assessment that for “true ” “exploratory” “studies, the evolutionary path may not be easily explained. No harm in asking?