The Poor Science of Medical Journals

[Guest guest]

So, as we broach the subject of Research and Outcome studies, let's

weigh in and humble ourselves and give great credence to our clinical

wonders, what we hold dear.

To the friends and collegues that went to jail as charlatains,

quacks, and hoodlums to be able to take care of their patients in

spite of the beliefs and " research " of the Medical and Pharmaceauticl

juggernaut.

Gee, was that a soapbox?? LOL

G.D. Smalling, D.C.

So, how strong, or weak is most Medical Reasearch? Dr.

of the JSRM (Journal of the Royal Society of Medicine):

[THE POOR SCIENCE OF MEDICAL JOURNALS]

" The science that underpins medicine is presented in journals, and

most journals can point to landmark studies that changed medicine.

The BMJ, for example, published some of the first studies in

anaesthesia, on the cause of malaria, and on linking cigarettes to

lung cancer. It also published one of the first randomized controlled

trial, which scientifically was a major development. The journals

might also be said to have become `more scientific' over the years as

the rigour of the methods of the studies they publish has improved.

Yet medical journals often contain poor science. Basic scientists who

work in biology and chemistry are regularly scornful of the, mostly,

applied science that appears in medical journals. The journals have,

for example, published many reports of treatments applied to single

cases and to series of cases, which rarely allow confident

conclusions because of the absence of controls. Journals have also

been part of what might be called an `unscientific' method of

encouraging treatments that seem to make anatomic, physiological, or

biochemical sense, without insisting that they be properly evaluated

in practice.

The history of medicine is littered with treatments that seemed to

make sense but which ultimately did more harm than good. Sir

Arbuthnot Lane, who was mercilessly parodied in Bernard Shaw's

The Doctor's Dilemma, removed the colons of Londoners who were

severely fatigued and rich enough to meet his high fees. The

operation was supposedly removing toxins. A tenth of his patients

were killed by the operation. I belong to a generation who had their

tonsils removed to no benefit. While my wife, when having our first

child in 1982, was given an enema and had her pubic hair shaved—

procedures which are unpleasant and of no benefit.

Medicine itself probably deserves most criticism for its unscientific

behaviour but journals are the major link between science and

practice. In recent year, journals have been severely criticized for

publishing studies that are scientifically weak (in that their

conclusions are not supported by their methods and data) and

irrelevant to practitioners (and so patients).

PEER REVIEW: MORE EVIDENCE OF HARM THAN BENEFIT

Peer review—i.e. asking peers of the authors of scientific studies to

review the studies critically before publication—is the process that

is supposed to ensure the scientific quality of journals. It is a

sacred process—and the phrase `peer reviewed journal' is supposed to

guarantee quality. But clearly peer review is deficient. Despite

being central to the scientific process it was itself largely

unstudied until various pioneers—including Lock, former

editor of the BMJ, and Drummond Rennie, deputy editor of JAMA—urged

that it could and should be studied. Studies so far have shown that

it is slow, expensive, ineffective, something of a lottery, prone to

bias and abuse, and hopeless at spotting errors and fraud. The

benefits of peer review have been much harder to establish. As Rennie

says, `If it was a drug it would never get onto the market'.

Nevertheless, no journal would dare to abandon peer review. Editors

are convinced—even though they are finding it had to prove—that peer

review is invaluable.

SCIENCE FOR THE UNSCIENTIFIC

Medical journals differ from scientific journals in that they are

mainly read not by scientists but by practising doctors. But are not

doctors also scientists? We are, in that when we were medical

students our heads were filled with anatomy, biochemistry, physiology

and (if we are under 40) by molecular biology; but such teaching does

not a scientist make. Most doctors feel uncomfortable describing

themselves as scientists. Most are not even trained to appraise

critically a scientific article.

Why then are they sent journals filled with increasingly complex

science, most of which depends on statistical methods that they do

not understand? It is, I think, a historical hangover, and there are

lots of data to show that doctors spend little time reading the

original research in journals. They are sensible not to do so. The

average doctor spends not much more than an hour a week on

professional reading. Thus, it does not make sense to spend most of

that time reading one complex study. Doctors, sensibly, should read

synoptic, educational material; mostly that is what doctors do. Often

they read this material not in journals but in what are

condescendingly known as `throwaways', i.e. newspapers that use

journalists to summarize complex material and which are free because

they are funded by pharmaceutical advertising. "