From the UN: Reproducibility, Replication, and Fraud in Scientific Research
Dr. Sylvain Ehrenfeld
IHEU and the National Ethical Service of the AEU
representative to the UN
Dr. Reba Goodman,
New York Society for Ethical Culture and
Department of Pathology at Columbia University
Reproducibility is the foundation of all scientific research. It is the standard by which scientific claims are evaluated. Biomedical research in the US is a $100 billion a year business. Yet, much of the current published data has not been and often cannot be replicated by others, even if it is published in so-called top flight peer-reviewed journals.
We know that modern medicine has improved and extended life; however, monitoring research is vital. The World Health Organization (WHO) is active in over-seeing this area because the safety of medicine is a global and moral responsibility. There is a bi-annual meeting of the WHO Expert Committee on Selection and Use of Essential Medicines to review the latest scientific evidence on the efficacy, safety, and cost-effectiveness of medicines, in order to revise and update the WHO Model List of Essential Medicine for both children and adults. Members selected for the Expert Advisory Panel are chosen to represent a wide range of geographical locations, different approaches, professional experience, and gender balance. The most recent meeting of the Expert Committee meeting was held in Accra, Ghana in March 2011.
Despite expert oversight, it is not unusual for scientists to be unable to replicate published scientific studies. There are also numerous cases of fraud. Lack of reproducibility, replication, and fraud in the scientific community has now reached the newspapers. (See Wall Street Journal
, Dec. 2, 2010 and The New Yorker
, December 13, 2010). The public is becoming more and more wary of believing any scientific reports, including global warming. The high level of skepticism in both the public and the scientific community has led to a special issue of the journal Science
(December 2, 2011) devoted to the examination of data replication and reproducibility.
Famous and respected scientists have admitted to making up data: psychologist Dr. Diederick Staple admitted fabricating data in dozens of studies and Dr. Jan Hendrick Schon, at Bell Labs in New Jersey, admitted to widespread fabrication and manipulation of data. In a case where faking has serious consequences, British researcher, Dr. Andrew Wakefield, falsified data linking childhood vaccinations to autism.
Competition for dwindling funding contributes to fraud. For example, in the five year span from 2002 to 2007, there was a 25% increase in the number of scientists competing for research funding and publication in journals. Researchers, working globally across all scientific fields, rush their findings into publication before completing a stringent examination of their data. Another fundamental issue is the requirement of journals for positive results; they will not publish negative results.
Many reasons are proposed for lack of replication including methodology for data analysis. For a clear picture of results the crucial issues at stake include selection of the population for a clinical trial and how much and/or little information about the clinical trial should be given to patients and controls. The issue of replication and reproducibility is especially important when bringing a drug to market. Before a drug can pass approval, pharmaceutical companies are required to test each drug on volunteers with a specific disease for which the drug was developed i.e. they select a population with as little variability in patient health as possible. Thus, these trials do not provide sufficient information for a wider and larger untested population with greater biological variability.
Furthermore, expectations and suggestions can influence the clinical response. This was demonstrated in a study for the treatment of migraines and tension headaches in which a placebo was as effective in the control group of patients as the drug being tested. The efficacy of a drug is mostly based on how much greater effect it shows than a placebo. Drug companies often manipulate such findings to show a "significant difference."
Scientific research is time consuming and requires great patience, and the highest of ethical standards since the ultimate goal of drug safety is at stake. The difficulties of scientific research are not to be underestimated and should not be manipulated for ideological purposes. Experiments must be carried out with rigor and honesty. When possible, all data must be coded so that it can be analyzed and evaluated in the absence of preconception.