I recently wrote a review of Ben Goldacre's latest book, Bad Pharma, for Medical Writing, the journal of the European Medical Writers Association. That review is reproduced below.
By Ben Goldacre, published by Fourth Estate,
2012. ISBN 978-0-00-735074-2 (paperback) 448 Pages. £13.99
Bad Pharma is the latest book by the well known
anti-quackery campaigner Ben Goldacre, and attempts to explain to us
that medicine is broken. Despite the title, he criticises not only
the pharmaceutical industry, but also regulators, doctors, academic
clinical researchers, ethics committees, and various other players in
the world of clinical research. His take home message (I don’t
think a spoiler alert is really needed here!) is that we simply can't
trust the evidence that we see about the efficacy and safety of drugs
in common use.
The book is divided into 6 chapters, which cover different aspects of the pharmaceutical industry. Chapter 1 is entitled ‘Missing data’, and describes at considerable length the important problem of publication bias. The take home message from this chapter is that we cannot assess the evidence for a particular drug if not all the trials on it are published, and, worse still, those that are not published tend to be different from the ones that are. Chapter 2 is a brief and well put-together description of the drug development process. Chapter 3, ‘Bad regulators’, does what it says on the tin, and explains the many ways in which Goldacre believes that drug regulation isn’t working. Chapter 4 talks about the design of individual clinical studies and how they can be flawed. Chapter 5 describes how pragmatic randomised trials could be (but very rarely are) incorporated into routine clinical practice. This seems a little out of place, as it is not really about ‘bad pharma’ at all, but is interesting nonetheless. Chapter 6, the longest chapter of all at over 100 pages, talks about marketing in the pharmaceutical industry.
Goldacre has a well earned reputation as a fearless debunker of dodgy scientific claims. His previous book, Bad Science, mercilessly took to pieces the dubious tricks played by various pedlars of pseudoscience. He regularly writes articles both on his own blog and for the popular media in which he rigorously dissects questionable claims, pointing to the flaws in the scientific and statistical methods used by those who make them.
So if you are familiar with Goldacre’s reputation, then you would expect that this book would be backed up with similarly rigorous scientific arguments. However, you would be disappointed.
Goldacre tells us at several places in the book (quite correctly) about the importance of using systematic reviews and being careful not to cherry-pick examples that back up a specific point, and promises to cite systematic reviews to make his points. Sadly, the reality of the way he presents his evidence does not live up to those fine promises. He certainly presents the results of some systematic reviews, but he is far from consistent in doing this. At one place he presents a single study, which is not a systematic review, but describes it as a systematic review anyway. In many places he does exactly what he warns against and cherry-picks unrepresentative cases to make a point. He sometimes ignores evidence that contradicts his message. The overall impression is that he decided from the start that he was going to tell as powerful a story as possible that the whole system of drug research is flawed, rather than attempting to follow the evidence in a scholarly manner.
It would, however, be a mistake to dismiss this book as being based on poor scholarship and therefore unworthy of our attention. Despite the shortcomings in his use of evidence, Goldacre does make some important points whose validity is not in doubt.
One such point is that much of the evidence on how well drugs work is not available to patients and prescribers: the problem of publication bias that he describes in the first chapter. Many attempts have been made to fix this problem, and most big pharma companies now commit to publishing all their trials, although Goldacre describes these efforts (without presenting evidence) as ‘fake fixes’.
Nonetheless, it would be overly optimistic to assume that every study that takes place is published, and until we can be sure that it is, then we all need to try harder to ensure more complete publication. Goldacre also makes the very good point that even if incomplete publication has now been fixed, there is still a mountain of studies that were done in the past and are still not published, even though their results are still relevant to today’s medical practice. So we should not consider the problem solved until it has been solved retrospectively as well.
Goldacre’s criticisms of the secrecy that surrounds the regulatory process are also very well made. He points out that it is unjustifiable that regulators have access to huge amounts of data on the drugs they approve, but do not publish them. It is hard to argue with this. I personally cannot think of any valid reason why regulators do not routinely make submission dossiers available via their websites, and we could all have far more confidence in the regulatory process, as well as know far more about the drugs that we use, if they did.
Medical writers will find some parts of the book frankly offensive. Goldacre seems to use the terms ‘medical writing’ and ‘ghostwriting’ interchangeably, completely ignoring the considerable efforts that EMWA and other medical writing organisations have made to combat ghostwriting in the medical literature. He describes professional medical writing thus: ‘They [pharmaceutical companies] pay professional writers to produce academic papers, following their own commercial specifications, and then get academics to put their names to them.’ This is a caricature of the work of the medical writer based on a few examples of bad practice mostly dating from the 1990s, and EMWA members will be acutely aware that this bears little resemblance to the way medical writing is practised in real life today, even though Goldacre describes ghostwriting papers for academics who have no input into them as ‘bread-and-butter activities’ of medical writing. If you are offended by this mischaracterisation of the medical writing profession, then I am not surprised. It is telling that Goldacre does not provide any evidence to back up this claim, other than to quote some old individual cases where some companies did not play by the rules in the past. This is cherry-picking of the worst kind: there is no evidence whatever that those kinds of abuses were common even back in the 1990s when most of them occurred, let alone today. It would be like making the claim that most doctors are serial killers, and backing it up with reference to Harold Shipman.
Of course, if you did claim that most doctors were serial killers, no-one would believe you, because most people are very familiar with who doctors are and what they do, and know that most of them are conscientious and caring individuals. Sadly, however, medical writing is not such a well known profession, and it is probably true that many people who read the book will not be familiar with what medical writers do, and so will simply believe Goldacre’s flagrant mischaracterisation of our profession.
The book is divided into 6 chapters, which cover different aspects of the pharmaceutical industry. Chapter 1 is entitled ‘Missing data’, and describes at considerable length the important problem of publication bias. The take home message from this chapter is that we cannot assess the evidence for a particular drug if not all the trials on it are published, and, worse still, those that are not published tend to be different from the ones that are. Chapter 2 is a brief and well put-together description of the drug development process. Chapter 3, ‘Bad regulators’, does what it says on the tin, and explains the many ways in which Goldacre believes that drug regulation isn’t working. Chapter 4 talks about the design of individual clinical studies and how they can be flawed. Chapter 5 describes how pragmatic randomised trials could be (but very rarely are) incorporated into routine clinical practice. This seems a little out of place, as it is not really about ‘bad pharma’ at all, but is interesting nonetheless. Chapter 6, the longest chapter of all at over 100 pages, talks about marketing in the pharmaceutical industry.
Goldacre has a well earned reputation as a fearless debunker of dodgy scientific claims. His previous book, Bad Science, mercilessly took to pieces the dubious tricks played by various pedlars of pseudoscience. He regularly writes articles both on his own blog and for the popular media in which he rigorously dissects questionable claims, pointing to the flaws in the scientific and statistical methods used by those who make them.
So if you are familiar with Goldacre’s reputation, then you would expect that this book would be backed up with similarly rigorous scientific arguments. However, you would be disappointed.
Goldacre tells us at several places in the book (quite correctly) about the importance of using systematic reviews and being careful not to cherry-pick examples that back up a specific point, and promises to cite systematic reviews to make his points. Sadly, the reality of the way he presents his evidence does not live up to those fine promises. He certainly presents the results of some systematic reviews, but he is far from consistent in doing this. At one place he presents a single study, which is not a systematic review, but describes it as a systematic review anyway. In many places he does exactly what he warns against and cherry-picks unrepresentative cases to make a point. He sometimes ignores evidence that contradicts his message. The overall impression is that he decided from the start that he was going to tell as powerful a story as possible that the whole system of drug research is flawed, rather than attempting to follow the evidence in a scholarly manner.
It would, however, be a mistake to dismiss this book as being based on poor scholarship and therefore unworthy of our attention. Despite the shortcomings in his use of evidence, Goldacre does make some important points whose validity is not in doubt.
One such point is that much of the evidence on how well drugs work is not available to patients and prescribers: the problem of publication bias that he describes in the first chapter. Many attempts have been made to fix this problem, and most big pharma companies now commit to publishing all their trials, although Goldacre describes these efforts (without presenting evidence) as ‘fake fixes’.
Nonetheless, it would be overly optimistic to assume that every study that takes place is published, and until we can be sure that it is, then we all need to try harder to ensure more complete publication. Goldacre also makes the very good point that even if incomplete publication has now been fixed, there is still a mountain of studies that were done in the past and are still not published, even though their results are still relevant to today’s medical practice. So we should not consider the problem solved until it has been solved retrospectively as well.
Goldacre’s criticisms of the secrecy that surrounds the regulatory process are also very well made. He points out that it is unjustifiable that regulators have access to huge amounts of data on the drugs they approve, but do not publish them. It is hard to argue with this. I personally cannot think of any valid reason why regulators do not routinely make submission dossiers available via their websites, and we could all have far more confidence in the regulatory process, as well as know far more about the drugs that we use, if they did.
Medical writers will find some parts of the book frankly offensive. Goldacre seems to use the terms ‘medical writing’ and ‘ghostwriting’ interchangeably, completely ignoring the considerable efforts that EMWA and other medical writing organisations have made to combat ghostwriting in the medical literature. He describes professional medical writing thus: ‘They [pharmaceutical companies] pay professional writers to produce academic papers, following their own commercial specifications, and then get academics to put their names to them.’ This is a caricature of the work of the medical writer based on a few examples of bad practice mostly dating from the 1990s, and EMWA members will be acutely aware that this bears little resemblance to the way medical writing is practised in real life today, even though Goldacre describes ghostwriting papers for academics who have no input into them as ‘bread-and-butter activities’ of medical writing. If you are offended by this mischaracterisation of the medical writing profession, then I am not surprised. It is telling that Goldacre does not provide any evidence to back up this claim, other than to quote some old individual cases where some companies did not play by the rules in the past. This is cherry-picking of the worst kind: there is no evidence whatever that those kinds of abuses were common even back in the 1990s when most of them occurred, let alone today. It would be like making the claim that most doctors are serial killers, and backing it up with reference to Harold Shipman.
Of course, if you did claim that most doctors were serial killers, no-one would believe you, because most people are very familiar with who doctors are and what they do, and know that most of them are conscientious and caring individuals. Sadly, however, medical writing is not such a well known profession, and it is probably true that many people who read the book will not be familiar with what medical writers do, and so will simply believe Goldacre’s flagrant mischaracterisation of our profession.
At 448 pages, Bad Pharma is a long book. It is
probably longer than it needs to be: Goldacre’s fondness for using
anecdotes about specific cases to make his point adds more to the
emotive qualities of the book than it does to the scientific data
presented.
EMWA members will probably not learn much from this book that is new to them. There are some good explanations of how drug development works, but this will already be familiar territory. The book does, however, provide much food for thought. Although Goldacre goes beyond the evidence and overstates his case in places, he does, as I mentioned earlier, still make some valid points. If any EMWA members who read this book are prompted to give some more thought to how they can help to ensure that the trials they work on are always published, then it will have fulfilled a useful function.
EMWA members will probably not learn much from this book that is new to them. There are some good explanations of how drug development works, but this will already be familiar territory. The book does, however, provide much food for thought. Although Goldacre goes beyond the evidence and overstates his case in places, he does, as I mentioned earlier, still make some valid points. If any EMWA members who read this book are prompted to give some more thought to how they can help to ensure that the trials they work on are always published, then it will have fulfilled a useful function.
Adam Jacobs
Dianthus Medical Limited
Dianthus Medical Limited
Find me on Twitter at @dianthusmed
