Bad Pharma: How Medicine is Broken, And How We Can Fix It. Ben Goldacre

which is that they would never get any benefit from the drug in normal use. The correct level of pain to record for them is, therefore, their pain at the beginning of the study, before they had any kind of treatment (if you’re interested, this is called ‘Baseline Observation Carried Forward’). The analysis was duly redone, properly, and a more modest, more accurate view of the benefits of the drug was produced. In this case, it turns out that using the ‘last observation’ method overestimated the improvement in pain by about a quarter.

      Here’s the catch. Four out of five of these trials were then published in the peer-reviewed academic literature, the place where doctors look for evidence on whether a drug works or not (one trial wasn’t published at all). Every single one of the published analyses used ‘Last Observation Carried Forward’, the dodgy method, the one that exaggerates the benefits of the drug. Not one of them acknowledges that ‘last observation’ is a technique that overstates these benefits.

      You can see why it is important that we have access to all the information we can possibly get on every drug trial: not only are some whole trials withheld from us, but there are often hidden flaws in the methods used. The devil is in the detail, and there are many dodgy trials, as we shall soon see, with flaws that may not be clear even in the academic papers, let alone in the thin and uninformative summaries from regulators. Furthermore, as we shall also see very shortly, there are often worrying discrepancies between the regulators’ summary documents and what actually happened in the trial.

      This is why we need to get hold of a more detailed document on each trial: something called the Clinical Study Report (CSR). These are long pieces of work, sometimes thousands of pages, but they are complete enough for the reader to reconstruct exactly what happened to all the participants; and they will let you find out where the bodies are buried. Drug companies give this study report to the regulator – though still only for formally licensed uses of the drug – so both have a copy, and both should be happy to hand it over.

      We will now see what happens when you ask them.

       Three: Regulators withhold study reports that they do have

      In 2007, researchers from the Nordic Cochrane Centre were working on a systematic review for two widely used diet drugs, orlistat and rimonabant. A systematic review, as you know, is the gold-standard summary of the evidence on whether a treatment is effective. These are life-saving, because they give us the best possible understanding of the true effects of a treatment, including its side effects. But doing this requires access to all of the evidence: if some is missing, especially if unflattering data is deliberately harder to obtain, we will be left with a distorted picture.

      The researchers knew that the trial data they were able to find in the published academic literature was probably incomplete, because negative trials are routinely left unpublished. But they also knew that the European Medicines Agency (EMA) would have much of this information, since the manufacturers of drugs are obliged to give the study reports to the regulator when trying to get them onto the market. Since regulators are supposed to act in the interests of patients, they applied to the EMA for the protocols and the study reports. That was in June 2007.

      In August, the EMA responded: it had decided not to give out the study reports for these trials, and was invoking the section of its rules which allows it to protect the commercial interests and intellectual property of drug companies. The researchers replied immediately, almost by return of post: there is nothing in the study reports that will undermine the protection of someone’s commercial interests, they explained. But if there was, could the EMA please explain why it felt the commercial interests of the drug companies should override the welfare of patients?

      We should pause for just one moment, and think about what the EMA is doing here. It is the regulator that approves and monitors drugs for the whole of Europe, with the aim of protecting the public. Doctors and patients can only make meaningful decisions about treatments if they have access to all the data. The EMA has that data, but has decided that the drug companies’ interests are more important. Having spoken to a lot of people in regulation, I can offer one small insight into what on earth they might be thinking. Regulators, in my experience, are preoccupied with the idea that they see all the data, and use it to make the decision about whether a drug should go on the market, and that this is enough: doctors and patients don’t need to see the data, because the regulator has done all that work.

      This misunderstands a crucial difference between the decisions made by regulators and the decisions made by doctors. Contrary to what some regulators seem to think, a drug is not either ‘good’ and therefore on the market, or ‘bad’ and therefore off it. A regulator makes a decision about whether it’s in the interests of the population as a whole that the drug should be available for use, at all, ever – even if only in some very obscure circumstance, infrequently and cautiously. This bar is set pretty low, as we shall see, and lots of drugs that are on the market (in fact, the overwhelming majority) are hardly ever used.

      A doctor needs to use the same information as that available to the regulator in order to make a very different decision: is this the right drug for the patient in front of me right now? The simple fact that a drug is approved for prescription doesn’t mean it’s particularly good, or the best. In fact, there are complex decisions to be made in each clinical situation about which drug is best. Maybe the patient has failed to get better on one drug, so you want to try another, from a different class of drugs; maybe the patient has mild kidney failure, so you don’t want to use the most popular drug, as that causes very occasional problems in patients with dodgy kidneys; maybe you need a drug that won’t interfere with other drugs the patient is taking.

      These complex considerations are the reason we are OK with having a range of drugs on the market: even if some of them are less useful overall, they might be useful in specific circumstances. But we need to be able to see all of the information about them, in order to make these decisions. It is not enough for the regulators to grandly state that they have approved a drug, and therefore we should all feel happy to prescribe it. Doctors and patients need the data just as much as regulators do.

      In September 2007 the EMA confirmed to the Cochrane researchers that it wasn’t going to share the study reports on orlistat and rimonabant, and explained that it had a policy of never disclosing the data given as part of a marketing authorisation. A serious problem had emerged. These weight-loss drugs were being widely prescribed throughout Europe, but doctors and patients were unable to access important information about whether they worked, how bad the side effects were, which was more effective, or any of a whole host of other important questions. Real patients were being exposed to potential harm, in everyday prescribing decisions, through this lack of information that was enforced by the EMA.

      The researchers went to the European Ombudsman with two clear allegations. Firstly, the EMA had failed to give sufficient reasons for refusing them access to the data; and secondly, the EMA’s brief, dismissive claim that commercial interests must be protected was unjustified, because there was no material of commercial interest in the trial results, other than the data on safety and effectiveness, which doctors and patients obviously need to access. They didn’t know it at the time, but this was the beginning of a battle for data that would shame the EMA, and would last more than three years.

      It took four months for the EMA to respond, and over the next year it simply reiterated its position: as far as it was concerned, any kind of information the disclosure of which would ‘unreasonably undermine or prejudice the commercial interests of individuals or companies’ was commercially confidential. The study reports, it said, might contain information about the commercial plans for the drug. The researchers responded that this was unlikely, but was in any case of marginal importance, as part of a much more important and pressing situation: ‘As a likely consequence of [the] EMA’s position, patients would die unnecessarily, and would be treated with inferior and potentially harmful drugs.’ They regarded the EMA’s position as ethically indefensible. More than that, they said, the EMA had a clear conflict of interest: this data could be used to challenge its summary views on the benefits and risks of these treatments. The EMA had failed to explain why doctors and patients having access to study reports and protocols should undermine anyone’s reasonable commercial interests, and why these commercial interests