2011 DemingDataandObservationalStudi
- (Young & Karr, 2011) ⇒ S Stanley Young, and Alan Karr. (2011). “Deming, Data and Observational Studies.” Significance, 8(3). doi:10.1111/j.1740-9713.2011.00506.x
Subject Headings: Repeatable Experiment, Observational Study.
Notes
Cited By
Quotes
Abstract
“Any claim coming from an observational study is most likely to be wrong.” Startling, but true. Coffee causes pancreatic cancer. Type A personality causes heart attacks. Trans-fat is a killer. Women who eat breakfast cereal give birth to more boys. All these claims come from observational studies; yet when the studies are carefully examined, the claimed links appear to be incorrect. What is going wrong? Some have suggested that the scientific method is failing, that nature itself is playing tricks on us. But it is our way of studying nature that is broken and that urgently needs mending, say S. Stanley Young and Alan Karr; and they propose a strategy to fix it.
1. Introduction
Science works by experiments that can be repeated; when they are repeated, they must give the same answer. If an experiment does not replicate, something has gone wrong. In a large branch of science the experiments are observational studies: we look at people who eat certain foods, or take certain drugs, or live certain lifestyles, and we seem to find that they suffer more from certain diseases or are cured of those diseases, or – as with women who eat more breakfast cereal – that more of their children are boys. The more startling the claim, the better. These results are published in peer-reviewed journals, and frequently make news headlines as well. They seem solid. They are based on observation, on scientific method, and on statistics. But something is going wrong. There is now enough evidence to say that many have long though: that any claim coming from an observational study is most likely to be wrong - wrong in the sense that it will not replicate if tested rigorously.
As long ago as 1988 (1,2) it was noted that there are contradicted results for case-control studies in 56 different topic areas, of which
…
When many questions are asked of the same data,
…
Producing at least one false positive becomes a near certainty unless the data analysis accounts for the multiple
…
References
,
| Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2011 DemingDataandObservationalStudi | S Stanley Young Alan Karr | Deming, Data and Observational Studies | 2011 |