Reversal of Established Medical Practices

[Guest guest]

Apropos of the discussion on vertebroplasty is the article below that just came out in JAMA. I think it is important to have a ready grasp of the concepts in here. FreemanJAMA. 2012;307(1):37-38.

doi: 10.1001/jama.2011.1960 Reversals of

Established Medical PracticesEvidence to Abandon Ship1.

Vinay Prasad,

MD; 2.

Adam Cifu,

MD; 3.

P. A.

Ioannidis, MD, DSc[+]

Author Affiliations1.

Author Affiliations: Department of

Medicine, Northwestern University, Chicago, Illinois (Dr Prasad); Department of

Medicine, University of Chicago, Chicago (Dr Cifu); and Stanford Prevention

Research Center and Departments of Medicine and Health Research and Policy,

Stanford University School of Medicine, Stanford, California (Dr Ioannidis). 1.

Corresponding Author: P. A. Ioannidis, MD, DSc, Stanford Prevention

Research Center, Stanford University School of Medicine, Stanford University,

Medical School Office Bldg, Room X306, 1265 Welch Rd, Stanford, CA 94305 (jioannid@...). Ideally, good medical practices are

replaced by better ones, based on robust comparative trials in which new

interventions outperform older ones and establish new standards of care. Often,

however, established standards must be abandoned not because

a better replacement has been identified but simply because what was thought to

be beneficial was not. In these cases, it becomes apparent that clinicians,

encouraged by professional societies and guidelines, have been using

medications, procedures, or preventive measures in vain. For example,

percutaneous coronary intervention performed for stable coronary artery disease

and hormone therapy prescribed for postmenopausal women cost billions of

dollars and supported the existence of entire specialties for many years.

Stable coronary artery disease accounted for 85% of all stenting in the United

States at the time of the Clinical Outcomes Utilizing Revascularization and

Aggressive Drug Evaluation (COURAGE) trial.1​ Large, well-designed randomized

trials that tested whether these practices improved major patient outcomes

revealed that patients were not being helped. Defenders of these therapies and

interventions wrote rebuttals and editorials and fought for their specialties,

but the reality was that the best that could be done was to abandon ship. How many established standards of medical care are wrong? It is not known. Medical practice has evolved out of

centuries of theorizing, personal experiences, bits of evidence, expert

consensus, and diverse conflicts and biases. Rigorous questioning of

long-established practices is difficult. There are thousands of clinical

trials, but most deal with trivialities or efforts to buttress the sales of specific

products. Given this conundrum,

it is possible that some entire medical subspecialties are based on little

evidence. Their disappearance probably would not harm patients and might help

salvage derailed health budgets. However, it is unlikely that specialists would

support trials testing practices that constitute their main source of income. Instead, the research community

performs studies of modest incremental value without even knowing whether the

basic standards of care are appropriate. Rarely, some investigators find the

courage to test established “truths†with large, rigorous randomized trials.

When this happens, empirical evidence suggests that “medical reversals†may be

quite common. In an evaluation of 35 trials that were published in

a major clinical journal in 2009 and that tested an established clinical

practice, 16 (46%) reported results consistent with current beneficial

practice, 16 (46%) reported evidence that contradicted current practice and

constituted a reversal, and another 3 (9%) were inconclusive.2 Perhaps high-profile general medical

journals are more prone to publish unusual results and less inclined to defend

a clinical practice or specialized turf than specialty journals. However, it is unlikely that the selection filter in favor of reversal publications

is stronger than the selection filter favoring the validation of standard of

care. The mere testing of a

standard of care generates interest because many standards of care are never

tested. In another evaluation of trials published in 3 major general medical

journals or high–impact factor specialty journals,3​ of the 39 most-cited randomized

trials published in 1990-2003 that found a significant benefit of a clinical

intervention, 9 (23%) found effects stronger than those found in subsequent

studies and 19 (49%) found results replicated in subsequent studies, but 11

(28%) remained largely unchallenged, with no large trial conducted on the same

question. Many medical reversals involve

conditions for which the standard of care has been promoted over the years

based primarily on pathophysiological considerations. Often one or more trials exist,

but they have not tested clinically relevant outcomes or have been biased. For

example, vertebroplasty—the injection of polymethylmethacrylate cement into

fractured bone—gained popularity in the early 2000s for the treatment of

osteoporotic fractures. Initial studies addressed the pathophysiology of this

therapy, delineated the technical skills required to optimally perform the

procedure, and furthered the discussion about the benefits of vertebroplasty. Claims of benefit were strongly contradicted in 2 randomized trials4,5​ that included a sham procedure, which alone might have been responsible

for pain relief.

Trials without sham control might continue to show benefit, but it is difficult

to justify performing invasive, expensive operations simply to obtain placebo

effects. Despite the evidence, many specialists will not

abandon the procedure. A study of vertebroplasty utilization at one institution

showed little reduction in referrals after publication of studies contradicting

current practice and in fact even showed that increasing percentages of

referred patients were offered the procedure.6Similarly, the results of COURAGE

have done little to improve optimal medical management of stable coronary

artery disease prior to invasive intervention. Stenting may not improve

mortality, but the procedure apparently diminishes angina.7​ However, more than 50 years ago, Cobb et al8 demonstrated that large improvements in pain with internal mammary artery

ligation were comparable to results obtained with a sham procedure. As is the

case with vertebroplasty, stenting performed in patients with stable disease is

probably widely used as an expensive placebo for pain control. The increasing use of surrogate end

points and short-term outcomes has also affected the credibility of clinical

trial results. Reversal of important research findings is more common when

pragmatic clinical outcomes are not used for initial approval and licensing of

interventions. For instance, bevacizumab exploited an accelerated approval

process for treatment of metastatic breast cancer. Approval was granted based

on preliminary data on disease progression, an end point that may not

necessarily translate into improved life expectancy or quality of life for

patients. After accrual of further data, on November 18, 2011, the US Food and

Drug Administration revoked its prior approval for this indication. Because medicine is in part a

statistically driven science, a certain amount of reversal of standards of care

is inevitable. However, what is currently tolerated is far greater than the

uncertainty of statistics. Reversal of established practices implies at least 3

grave consequences besides unjustified cost. First, patients who undergo the

therapy during the years it is in favor receive all the risk of treatment and,

ultimately, no real benefit. Second, contradicting studies do not immediately

force a change in practice; the contradicted practice continues for years.9​ Third, contradiction of mainstream practices undermines trust in the

medical system. Given the slow rate of abandonment of

ineffective medical practices, the standards governing drug and device approval must be

strengthened. This means that newly proposed innovations should be evaluated in

sufficiently large randomized trials that demonstrate improvement in important

clinical end points before being widely disseminated. Such an insistence on

well-designed, large studies may be seen as overly costly during times of

financial hardship. However, the costs of permitting widespread use of

ineffective interventions are much greater. In the case of vertebroplasty, a few

million dollars used to conduct a proper clinical trial before regulatory

approval might have saved nearly a billion dollars a year over the course of a

decade.10 For unnecessary hormone therapy and coronary stenting, the cost has been

even greater. Large trials of new

innovations should be designed and conducted by investigators without conflicts

of interest, under the auspices of nonconflicted scientific bodies. Instead of

designing, controlling, and conducting the trials, manufacturers may offer the

respective budget to a centralized public pool of funding, keeping the trial

design and conduct independent. Asking corporate sponsors to conduct pivotal trials

on their own products is like asking a painter to judge his or her own painting

so as to receive an award. If a manufacturer can be allowed to manipulate the

system to create a blockbuster product from an ineffective drug, the temptation

is hard to resist. Besides the need for better evidence

for new interventions, medical reversals also suggest that reality checks should

be encouraged for established practices that constitute the core of medical

care. Priority should be given to testing practices having limited or no prior

randomized evidence for their use, reassessing old evidence that may no longer

be relevant for current clinical settings, and evaluating therapies and

interventions that are most expensive. If almost half of these practices are

wrong, as empirical studies suggest,2​ the principle of equipoise is fully

satisfied and randomization is indicated.

Author Information

1.

Author Affiliations: Department of Medicine, Northwestern University, Chicago, Illinois (Dr

Prasad); Department of Medicine, University of Chicago, Chicago (Dr Cifu); and

Stanford Prevention Research Center and Departments of Medicine and Health

Research and Policy, Stanford University School of Medicine, Stanford,

California (Dr Ioannidis). Corresponding Author: P. A. Ioannidis, MD, DSc,

Stanford Prevention Research Center, Stanford University School of Medicine,

Stanford University, Medical School Office Bldg, Room X306, 1265 Welch Rd,

Stanford, CA 94305 (jioannid@...). Conflict of Interest Disclosures: All authors have completed and

submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and

none were reported.

REFERENCES

1.

1. 1.

Boden WE, 2.

O’Rourke RA, 3.

Teo KK, 4.

et al., 5.

COURAGE Trial Research Group. Optimal medical therapy with or without PCI for stable

coronary disease. N Engl J Med. 2007;356(15):1503–1516, pmid:17387127.CrossRefMedline2.

2. 1.

Prasad V, 2.

Gall V, 3.

Cifu A. The frequency of medical reversal. Arch Intern Med.

2011;171(18):1675–1676, pmid:21747003.Free Full

Text3.

3. 1.

Ioannidis JP. Contradicted and initially stronger effects in highly

cited clinical research. JAMA. 2005;294(2):218–228, pmid:16014596.Free Full

Text4.

4.