Disease of flawed insights
11 March 2015 11:03 PM GMT
Our bodies and minds are prisoners of modern science. Not a day goes by without newspapers, TV channels and the Internet plying us with the latest medical “insights” into the workings of maladies such as diabetes, or the promising results of a clinical trial of a drug for Alzheimer’s.
As most such studies are published in peer-reviewed journals of repute, even peers and practising physicians trust them, let alone the lay public. However, the truth is shockingly contradictory. In the last one decade, a series of metastudies (study of studies) has revealed that a large number of studies, especially in medical science and psychology, are either shoddy or statistically manipulated, and hence false.
John Ioannidis, an epidemiologist at Stanford University, was the first to blow the whistle in his provocatively titled 2005 paper, Why Most Published Research Findings are False. He demonstrated that in studies characterised by researcher prejudice, greater flexibility in design and technique, high glamour quotient, and high financial stakes (all of which is true of most studies), the researcher is likely to come up with an erroneous result.
In 2012, researchers at a US biotech firm, Amgen, reported that they could replicate no more than six of the 53 “landmark” studies in cancer research. Likewise, in 2011, researchers at Bayer found they could not verify more than 20-25 per cent of the research papers on cancer, cardiovascular and women’s health studies. Ominously, between 2000 and 2010, no fewer than 80,000 patients enrolled in clinical trials based on research that was later retracted because of errors or fraud.
Arguably, that is only the tip of the iceberg. Most drug companies are extremely secretive about data on clinical trials. Ben Goldacre, a British doctor and author of the bestseller Bad Science, is campaigning for penalising them if they don’t make full disclosure.
To verify, and not simply trust, a research finding is the lynchpin of modern science’s prerogative to objective truth. That scientists are not interested in replicating the findings of their peers is behind the current crisis of fraudulent research.
Statistical errors, inadvertent or otherwise, is just one of many factors that perpetuate bad science. Peer reviewers have been found guilty of oversight when it comes to spotting errors. To make matters worse, professional rivalry coupled with competition for a shrinking pool of funds and jobs encourages a publish-or-perish culture, which clearly affects the quality of papers published.
There is danger that some of these shoddy scientific claims might get reflected in medical practice. Besides, it also means that precious funds, a significant amount of which comes from the taxpayers’ pocket, are squandered on bad or useless research. Alarmed by the scale of the problem, some organisations have begun to put replication back on the table. Notably, in 2012, PLoS ONE, an open access journal, launched the Reproducibility Initiative, a service through which life scientists can get their work verified by an independent lab for a fee.
Unfortunately, even as India expands its medical research footprint, there are no signs of alarm over this worrisome issue. How much of Indian medical research is verified is anybody’s guess. But it’s high time the scientific fraternity started a public debate on this subject that has bearing not only on the quality of research but also on the lives of millions of Indians.
Ioannidis is all for making it a public affair but he is not very sanguine that this will happen quickly, though. He told The Atlantic magazine: “It’s difficult to change the way that everyday doctors, patients, and healthy people think and behave.” Hope he’s wrong. DOWN TO EARTH
As most such studies are published in peer-reviewed journals of repute, even peers and practising physicians trust them, let alone the lay public. However, the truth is shockingly contradictory. In the last one decade, a series of metastudies (study of studies) has revealed that a large number of studies, especially in medical science and psychology, are either shoddy or statistically manipulated, and hence false.
John Ioannidis, an epidemiologist at Stanford University, was the first to blow the whistle in his provocatively titled 2005 paper, Why Most Published Research Findings are False. He demonstrated that in studies characterised by researcher prejudice, greater flexibility in design and technique, high glamour quotient, and high financial stakes (all of which is true of most studies), the researcher is likely to come up with an erroneous result.
In 2012, researchers at a US biotech firm, Amgen, reported that they could replicate no more than six of the 53 “landmark” studies in cancer research. Likewise, in 2011, researchers at Bayer found they could not verify more than 20-25 per cent of the research papers on cancer, cardiovascular and women’s health studies. Ominously, between 2000 and 2010, no fewer than 80,000 patients enrolled in clinical trials based on research that was later retracted because of errors or fraud.
Arguably, that is only the tip of the iceberg. Most drug companies are extremely secretive about data on clinical trials. Ben Goldacre, a British doctor and author of the bestseller Bad Science, is campaigning for penalising them if they don’t make full disclosure.
To verify, and not simply trust, a research finding is the lynchpin of modern science’s prerogative to objective truth. That scientists are not interested in replicating the findings of their peers is behind the current crisis of fraudulent research.
Statistical errors, inadvertent or otherwise, is just one of many factors that perpetuate bad science. Peer reviewers have been found guilty of oversight when it comes to spotting errors. To make matters worse, professional rivalry coupled with competition for a shrinking pool of funds and jobs encourages a publish-or-perish culture, which clearly affects the quality of papers published.
There is danger that some of these shoddy scientific claims might get reflected in medical practice. Besides, it also means that precious funds, a significant amount of which comes from the taxpayers’ pocket, are squandered on bad or useless research. Alarmed by the scale of the problem, some organisations have begun to put replication back on the table. Notably, in 2012, PLoS ONE, an open access journal, launched the Reproducibility Initiative, a service through which life scientists can get their work verified by an independent lab for a fee.
Unfortunately, even as India expands its medical research footprint, there are no signs of alarm over this worrisome issue. How much of Indian medical research is verified is anybody’s guess. But it’s high time the scientific fraternity started a public debate on this subject that has bearing not only on the quality of research but also on the lives of millions of Indians.
Ioannidis is all for making it a public affair but he is not very sanguine that this will happen quickly, though. He told The Atlantic magazine: “It’s difficult to change the way that everyday doctors, patients, and healthy people think and behave.” Hope he’s wrong. DOWN TO EARTH
Next Story
