Most Science Studies Appear to Be Tainted By Sloppy Analysis

September 15, 2007

Wall Street Journal article:

" Take the discovery that the risk of disease may vary between men and

women, depending on their genes. Studies have prominently reported

such sex differences for hypertension, schizophrenia and multiple

sclerosis, as well as lung cancer and heart attacks. In research

published last month in the Journal of the American Medical

Association, Dr. Ioannidis and his colleagues analyzed 432 published

research claims concerning gender and genes.

Upon closer scrutiny, almost none of them held up. Only one was

replicated.

Statistically speaking, science suffers from an excess of

significance. Overeager researchers often tinker too much with the

statistical variables of their analysis to coax any meaningful insight

from their data sets. " People are messing around with the data to find

anything that seems significant, to show they have found something

that is new and unusual, " Dr. Ioannidis said. "

Recommended Posts