What’s wrong with evidence-based medicine?

By Devorah E. Klein, Gary Klein, and Shawna J. Perry

Three reasons to be wary include issues of trust, shifting conditions, and flawed research

Every health-care system balances coverage, quality, and cost – often focusing on one or two at the expense of the others. European systems, for example, tend to focus on coverage and ensure universal access to care; in the United States, by contrast, quality appears to be paramount.

But, whatever one’s view is about the proper priorities, it is clear that the US has room for improvement in all three areas. Fortunately, efforts are underway to address each of them. And efforts in the US have implications for other countries as well.

The US Patient Protection and Affordable Care Act (the landmark legislation now widely known as “Obamacare”) aims to widen coverage, while so-called accountable care organisations, such as Kaiser Permanente, try to contain costs by aligning the interests of providers and payers. However, efforts to raise quality through the application of evidence-based medicine (EBM) risk ignoring what we know about human cognition and expertise, and may undermine the vital role played by a physician’s expert judgment.

There are at least three reasons to be wary. For starters, EBM is grounded in a fundamental distrust of physicians’ intuition – that is, the rendering of pattern-based judgments that rely on years of experience. To be sure, there are good reasons to be skeptical of intuition, given the countless cases in which it has proved to be wrong. But it does not follow that all cases of intuition are flawed or that skilled intuition has no function in healthcare.  Physicians develop expertise over many years, and when they have ample opportunities for feedback about their judgments, their intuitions are valuable, particularly in more complex cases. An experienced physician will evaluate a patient’s vital signs and test results in the context of the patient’s life – for example, whether he is an octogenarian with diabetes and a smoker’s cough, or a three-week premature baby. With heuristics, intuition, and experience, an experienced physician can best understand a complicated case and develop a plan.

In fact, an experienced doctor’s conclusions may be more accurate than those provided by EBM. That is because EBM, though based on data from randomised trials and rigorous experiments, is designed for situations that approximate the conditions of patients in those tests. The problem is that when the context changes, the trial results become less reliable.

In such cases, the physician must determine how close his or her patient’s situation matches that in the relevant studies. If the disease is not as advanced as in the trials, the doctor will have to decide whether to embark on the recommended protocol. And clinicians may be unsure about which constraints are inviolate. Ultimately, the final judgment is informed by personal experience and pattern recognition, an approach that EBM supporters discount or even disparage.

A second problem with EBM is that it offers little guidance when a medical condition is evolving. For example, acute asthma may be the focus of care at one moment, but might shift to the patient’s diabetes later. EBM guidelines focus on asthma treatment or diabetes, but not both, or even how they may interact and change over time.

Finally, and perhaps most critically, one must ask how clinicians are to make decisions when there are gaps in the EBM knowledge base. Doctors often spot trends and develop hypotheses that involve guesswork and that are validated by trials only later. Should doctors therefore ignore observed patterns until the data are in? Indeed, one might wonder how such trials are initiated at all if clinicians do not tentatively explore a problem in the first place. To insist that all treatment decisions be based on existing best practices stifles this exploration and prevents potential medical breakthroughs.

Advocates of EBM respond that it is the job of researchers to generate the data that are turned into best practices; the clinician’s role is to implement the results. But this approach runs counter to medical history, in which advances come only after practitioners notice anomalies, discover flaws in current “best practices,” or improve on existing approaches. Many medical advances, from ulcer therapies to joint replacement, have evolved through clinician curiosity, not as a result of trials.

Moreover, important sources of EBM have themselves proved to be misleading. The Framingham Heart Study, considered the gold standard in its field, was shown to be flawed, owing to its focus on white males. For example, a key symptom of a heart attack identified by the study (the feeling of “an elephant sitting on my chest”) is found in only 5% of women. As one emergency medicine physician ruefully admitted to us, “I think about all the women I sent home to their deaths because I was following best practices.”

As we consider a likely future in which physicians adhering to EBM are paid more, we must consider the cognitive limitations and the human cost of unquestioning compliance with so-called “best practices.” A more effective approach must be to combine EBM with the expertise and intuition of experienced caregivers, and to take the benefits of both.

****

Devorah E. Klein, a Principal with Continuum, is a cognitive psychologist working to design medical products, systems, and services. Gary Klein, is a senior scientist with MacroCognition LLC, and the author of Seeing What Others Don’t: The Remarkable Ways We Gain Insights. Shawna J. Perry is Director for Patient Safety Systems Engineering for Virginia Commonwealth University Health Systems and a professor of emergency medicine at VCU School of Medicine.