Publication Bias—the Hidden Systematic Flaw in Medicine that Can Threaten Your Life

February 13, 2013

By Dr. Mercola

Exposure and concerns over medical research fraud is on the rise, and for good reason. Publication bias — the practice of selectively publishing trial results that serve an agenda — represents a “systematic flaw of the scientific basis of medicine,” Ben Goldacre says in his TED Talk above.

Indeed this issue is a very serious one, and I agree with Goldacre’s assertion that it is “a cancer at the core of evidence-based medicine.” It undermines and negates everything modern medicine prides itself in.

In recent years, we’ve discovered that fraud can occur anywhere. Even at the best institutions.

Even if a drug or treatment is “scientifically proven,” the examples highlighted by Goldacre make it crystal clear that this simply is not a guarantee of safety or effectiveness.

Likewise, if an alternative treatment has not been published in a medical journal, it similarly does not mean it is unsafe or ineffective. There's much to be said for the ancient tried-and-true remedies, even if they've not been rigorously studied by a modern researcher.

What Gets Published, and What Doesn’t?

I am a firm believer in the scientific method, provided it's applied appropriately. And that's the key issue here. In order to qualify in the first place, the research must be unbiased, unprejudiced and free from any significant conflicts of interest. Sadly, this is simply not the case with most of modern medicine — especially not when it comes to drug research. It is clearly the exception and not the rule.

In the talk above, and in two recent articles published in Salon¹ and The New York Times,² Goldacre offers a number of examples of publication bias, which is pervasive across all fields of medicine. Both articles are excellent, and I recommend reading both of them if you have the time, along with his 13-minute long talk above. Bias can take a few different forms, such as:

• Positive results are published while negative ones are not
• Retractions are frequently not published, which would alert prescribers, researchers and academia that a study has been retracted due to fraud or other wrongdoing and cannot be relied upon
• The source of funding predictably affects the outcome of the trial

For example:

• Astonishingly, half of all clinical trials ever completed on the medical treatments currently in use have never been published in the medical literature. Trials with positive results for the test treatment are about twice as likely to be published, and this applies to both academic research and industry studies
• Former drug company researcher Glenn Begley looked at 53 papers in the world's top journals, and found that he and a team of scientists could NOT replicate 47 of the 53 published studies — all of which were considered important and valuable for the future of cancer treatments³
• In 2010, three researchers from Harvard and Toronto identified all the published trials for five major classes of drugs, and then measured two key features: Were they positive, and were they funded by industry? Out of a total of 500 trials, 85 percent of the industry-funded studies were positive, compared to 50 percent of the government-funded trials
• In 2007, researchers identified all published trials of cholesterol-lowering drugs known as statins. A total of 192 trials were found in which either two statins were compared to each other, or a statin was compared against a different kind of treatment. Industry-funded studies were 20 times more likely to favor the test drug, compared to those with independent funding
• According to a 2011 study in the Journal of Medical Ethics,⁴ nearly 32 percent of retracted papers were not noted as having been retracted by the journal in question, leaving the readers completely in the dark about the inaccuracies in those studies

The Industry Doesn’t Pay for Negative Results

Scientific fraud and/or the misuse of science to further a preconceived commercial agenda is so rampant today that it can be quite tricky to determine what's what. One key factor you'd be wise to consider is who paid for the study?

It's well-established that the source of funding can significantly skew research results, as those who pay generally want the research to be of benefit to them, one way or another. Truly independent research that is not funded or executed by any person or group with a financial stake or interest in the results is, generally speaking, the most trustworthy. Although sometimes you may have to do some sleuthing to determine whether the research might have hidden ties or agendas.

In the New York Times, Goldacre writes:

“There is one last study I’d like to tell you about. It turns out that this pattern of industry-funded trials being vastly more likely to give positive results persists even when you move away from published academic papers and look instead at trial reports from academic conferences, where data often appears for the first time...

Fries and Krishnan studied all the research abstracts presented at the 2001 American College of Rheumatology meetings that reported any kind of trial and acknowledged industry sponsorship in order to find out what proportion had results that favored the sponsor’s drug. There is a small punchline coming, and to understand it we need to talk a little about what an academic paper looks like. In general, the results section is extensive: The raw numbers are given for each outcome and for each possible causal factor, but not just as raw figures... each detail of the result is described in table form and in shorter narrative form in the text, explaining the most important results. This lengthy process is usually spread over several pages.

In Fries and Krishnan [2004], this level of detail was unnecessary. The results section is a single, simple and — I like to imagine — fairly passive-aggressive sentence:

'The results from every RCT (45 out of 45) favored the drug of the sponsor.'

This extreme finding has a very interesting side effect for those interested in time-saving shortcuts. Since every industry-sponsored trial had a positive result, that’s all you’d need to know about a piece of work to predict its outcome: If it was funded by industry, you could know with absolute certainty that the trial found the drug was great.”

When Negative Results Go 'Missing in Action'

Publication bias is profoundly serious, because the end result is that people frequently will die if they are making choices on inaccurate information and recommendations. Research does not exist in a vacuum. Published studies are used by doctors and health agencies as the basis for making recommendations and writing prescriptions. When they’re given a radically skewed picture of the facts, how can they make sound recommendations?

According to Goldacre, negative results missing in action cuts to the core of publication bias. When negative results are suppressed, people die. Sometimes in very large numbers.

In 1980, a study was done on a heart arrhythmia drug called lorcainaide. It included 100 people. Half of them received the drug; the other a placebo. Among those who received the drug, 10 died, compared to just one death in the placebo group. The trial was stopped and the drug was abandoned. The results of the study were never published. Over the next decade, other pharmaceutical companies created and marketed similar drugs to treat arrhythmia in heart attack patients. An estimated 100,000 people died before the deaths were finally traced back to the drugs. This case is now used as a perfect example of the price of publication bias, as the publication of those negative results could have provided an early warning.

Currently, the science behind the flu drug Tamiflu is also M.I.A., which should concern every citizen in every country that recommends it and stockpiles it in anticipation of a flu pandemic. Remarkably, eight of the 10 studies on Tamiflu have never been released for review, despite years of effort.

The Cochrane Collaboration conducts and publishes analyses of the scientific evidence supporting the use of various drugs and vaccines. They are considered the "gold standard" of independent scientific reviews, so when they issue a report, you're well advised to pay heed because it's free of conflict of interests and therefore very objective. I've previously discussed a number of their reviews on flu vaccines that have shed light on the sheer lack of scientific data supporting the claim that flu vaccines are a safe and effective means of preventing seasonal influenza.

Last year, Cochrane decided to update previous reviews that might have a bearing on influenza management, which includes Tamiflu. The previous assessment of Tamiflu was done in 2009. At the time, the group was unable to get Roche, the manufacturer of Tamiflu, to release eight of the 10 clinical trials involving the drug. The review therefore concluded that:

"Paucity of good data has undermined previous findings for oseltamivir's prevention of complications from influenza. Independent randomized trials to resolve these uncertainties are needed."

Years later and faced with continued stonewalling, Cochrane in collaboration with the British Medical Journal decided to take the issue to the public. The BMJ Open Data Campaign⁵ was recently created in an effort to force transparency as Roche continues to refuse to release the data from eight out of 10 clinical trials on Tamiflu. The campaign site contains links to Cochrane's correspondence with not just Roche, but also with the CDC and WHO — all of whom appear to be complicit in this scheme to massively promote a drug without scientific support for doing so.

The Depressing Research on Antidepressants

The antidepressant Reboxetine is another example of how negative results jeopardize the health of patients. Goldacre had a patient who found no relief from other antidepressants, so after doing his research, he put the patient on Reboxetine. However, they’d both been sorely misled.

As it turns out, while seven studies had been completed on the drug, only ONE was published — the one showing a favorable result. Six studies showing negative results were never published, and were therefore not taken into consideration when Goldacre suggested the drug to his patient. Three trials comparing Reboxetine against other antidepressants had also been published, in which the drug was found to be as just as good as the others. However, three times as many patients’-worth of data was collected, but never published, showing that Reboxetine was worse than other antidepressants, and that patients suffered more debilitating side effects on it.

Goldacre writes:⁶

“I did everything a doctor is supposed to do. I read all the papers, I critically appraised them, I understood them and I discussed them with the patient. We made a decision together, based on the evidence. In the published data, reboxetine was a safe and effective drug. In reality, it was no better than a sugar pill, and worse, it does more harm than good. As a doctor, I did something which, on the balance of all the evidence, harmed my patient, simply because unflattering data was left unpublished...

The repercussions of this go way beyond simply misleading doctors about the benefits and harms of interventions for patients, and way beyond trials. Medical research isn’t an abstract academic pursuit: It’s about people, so every time we fail to publish a piece of research we expose real, living people to unnecessary, avoidable suffering.”

Publication bias has been well studied — over 100 of them have been done. And they all testify to the fact that publication bias is very real, and very serious. For example, researchers looked at all trials submitted to the FDA during the approval process of 12 different antidepressants. They found 38 positive results, and 36 negative ones. That’s just about 50/50 going either way. But guess how many of these studies could be found in the published medical literature after the drugs were approved? Thirty-seven of the positive studies were published, and only THREE of those with negative findings. This is a staggering difference, and this publication bias is no doubt having a profoundly negative impact on patients.

Fake Fixes Frequently Fatal

So, is anything being done to remedy this pervasive, ongoing problem? In the New York Times, Goldacre writes:

“The Food and Drug Administration Amendments Act of 2007 is the most widely cited fix. It required that new clinical trials conducted in the United States post summaries of their results at clinicaltrials.gov within a year of completion, or face a fine of $10,000 a day. But in 2012, the British Medical Journal published the first open audit of the process, which found that four out of five trials covered by the legislation had ignored the reporting requirements. Amazingly, no fine has yet been levied.

An earlier fake fix dates from 2005, when the International Committee of Medical Journal Editors made an announcement: their members would never again publish any clinical trial unless its existence had been declared on a publicly accessible registry before the trial began. The reasoning was simple: if everyone registered their trials at the beginning, we could easily spot which results were withheld; and since everyone wants to publish in prominent academic journals, these editors had the perfect carrot. Once again, everyone assumed the problem had been fixed.

But four years later we discovered, in a paper from The Journal of the American Medical Association, that the editors had broken their promise: more than half of all trials published in leading journals still weren’t properly registered, and a quarter weren’t registered at all.

Even if these fixes had been successful, we would still be decades away from knowing the full truth about our medical treatments, because today’s decisions are informed by the trials of the past, on drugs that were first researched and approved in 2007, 2002, 1998 and earlier. None of the reforms has even tried to ensure public access for these results, and so they remain buried in dry storage archives, deep underground.”

Real Fixes are What is Needed

The last point is an important one. In order for modern medicine to be “science-based” we have to go far enough back to include ALL the research for ALL the treatments that are currently IN USE. Otherwise, we’re operating on quicksand. As Goldacre states in his TED talk:

“I don’t know what world it is in which we’re only practicing medicine based on trials completed in the past two years.”

Fixing the problem and returning science-based medicine is easy though. According to Goldacre, we must:

1. Publish all human trials, including older trials, for all drugs in current use
2. Tell everyone you know about this problem. For more information, contact info@alltrials.org

Take Control of Your Health

It's important to understand that our current medical system has been masterfully orchestrated by the drug companies to create a system that gives the perception of science when really it is a heavily manipulated process designed to convince and deceive you to use expensive and potentially toxic drugs that benefit the drug companies more than they benefit your health. Across the board, drugmakers do an excellent job of publicizing the findings they want you to know, while keeping studies that don’t support their product hidden from you and the medical community.

It's important to realize that all research is NOT published. And it should come as no surprise that drug studies funded by a pharmaceutical company that reaches a negative conclusion will rarely ever see the light of day...

What this means is that even if you scour the medical literature to determine what the consensus is on any given medical topic, what you'll find is an overwhelming preponderance of data in favor of the drug approach that in no way, shape or form reflects the reality of the scientific investigation that went into that specific drug. With so much data missing in action, what does the claim “scientifically proven” really amount to? It certainly cannot be construed as a guarantee of safety or effectiveness...

I recommend using all the resources available to you, including your own common sense and reason, true experts' advice and experiences of others, to determine what medical treatment or advice will be best for you in any given situation.

© Copyright 1997-2013 Dr. Joseph Mercola. All Rights Reserved.

http://articles.mercola.com/sites/articles/archive/2013/02/13/publication-bias.aspx?e_cid=20130213_DNL_art_1&utm_source=dnl&utm_medium=email&utm_campaign=20130213