Why a treatment that works in trials can fail in practice

A drug posts a strong result in a randomised trial, wins approval, and then underperforms in the clinic. Doctors notice that the patients in front of them are older, take four other medicines, miss doses, and arrive late in the illness, none of which happened in the trial. The efficacy-effectiveness gap is the name for that shortfall: the effect measured under ideal conditions does not transfer to ordinary patients, because the trial removed exactly the messiness that ordinary care contains.

What everyone sees

What everyone sees is either a bad drug or bad doctors. If the trial showed a benefit and the clinic does not, the obvious readings are that the drug was oversold or that real-world prescribing is sloppy. Both readings treat the trial result as a general truth about the medicine and the clinic as a noisy version of the trial. Nobody asks what the trial excluded, because exclusions are reported in a methods table rather than in the headline. The gap looks like a failure of execution rather than a predictable consequence of who was allowed into the study.

What is actually happening

Peter Rothwell set out the problem in The Lancet in 2005, arguing that trial results apply only to the patients who would have qualified for the trial, and that this population is often unrepresentative of routine practice. Singal, Higgins, and Waljee described the distinction between efficacy trials, which test whether an intervention can work under ideal conditions, and effectiveness trials, which test whether it does work under usual conditions. Kennedy-Martin and colleagues reviewed the representativeness of randomised trial samples in Trials in 2015 and found that trial participants are typically younger, healthier, and less likely to be from minority or low-income groups than the patients who eventually receive the treatment. The mechanism is not fraud but selection: adherence is monitored, comorbidity is excluded, co-medication is restricted, and support is provided. Remove those supports and the measured effect shrinks, sometimes to nothing.

Why it stays hidden

It stays hidden because both numbers are true and they are reported by different institutions. The trial result belongs to the regulator and the journal; the clinic result belongs to the physician and the patient, and the two are rarely placed side by side. Adherence is also invisible in aggregate: a patient who takes half the doses looks like a treatment failure rather than a delivery failure. Finally, the gap is most severe in exactly the populations least represented in trials, so the people most likely to experience it are the least likely to appear in the evidence that was supposed to cover them.

How the efficacy-effectiveness gap shows up in ordinary care

Chronic disease is the clearest case. A diabetes or blood-pressure drug tested in a monitored trial population meets patients who forget doses, cannot afford refills, and manage several conditions at once, and the observed benefit is smaller.

Mental health and oncology show the same pattern. Trials recruit patients well enough to consent and attend, while routine care includes frailty, cognitive impairment, and social instability, all of which reduce the effect that was measured.

The evidence in numbers and field studies

Rothwell’s Lancet paper in 2005 catalogued how trial exclusion criteria systematically remove older patients, those with comorbidity, and those taking interacting drugs, which limits the external validity of the result.

Kennedy-Martin and colleagues, writing in Trials in 2015, reviewed the literature and found consistent under-representation of older adults, women in some disease areas, and ethnic minorities, with the result that the treated population differs from the tested one in ways that matter for effect size.

Pragmatic trials, which randomise within routine care and accept the messiness, are the standard remedy. They typically report smaller effects than explanatory trials of the same intervention, which is the gap made visible.

When the gap is small and when it is dangerous

The gap is modest when the intervention is simple, the population is stable, and adherence is easy, as with a single-dose vaccine or a short antibiotic course. It is largest for long-term behavioural treatments and for drugs whose benefit depends on continuous use.

It becomes dangerous when guidelines are written from trial data alone and applied to patients the trials never enrolled. The honest response is not to distrust trials but to ask who was in them, and to prefer pragmatic evidence when the question is what happens in ordinary practice.

Questions readers ask

What is the efficacy-effectiveness gap?

It is the difference between how well an intervention performs under the ideal conditions of a randomised trial and how well it performs in routine practice, where adherence, comorbidity, and support differ from the trial setting.

Why do drugs that work in trials fail in real life?

Because trials exclude the patients and conditions that reduce benefit. Participants are typically younger and healthier, doses are monitored, and co-medication is restricted, so removing those supports shrinks the measured effect.

What is the difference between efficacy and effectiveness?

Efficacy is whether an intervention can work under ideal conditions, usually tested in an explanatory randomised trial. Effectiveness is whether it does work under usual conditions, usually tested in a pragmatic trial or observed in routine care.

How can the gap be reduced?

By running pragmatic trials inside routine care, by broadening eligibility so trial populations resemble the treated population, and by reading the methods table to see who was excluded before applying a result.