The scientific literature has recognised a number of biases that affect randomised controlled trials. In general these tend to magnify the benefit of a drug and minimise adverse reactions. All these biases lead to poor innovation.
Abuse of placebo
It is obvious that a comparison with placebo rather than with an active comparator allows better appraisal of a drug’s benefit. In this respect there is considerable disagreement between the Declaration of Helsinki and the rules issued by the US and European regulatory agencies. There are three different types of abuse of placebo:
- direct comparison with placebo (two-arm studies)
- use of placebo in an add-on design
- the three-arm design (placebo, comparator, new drug).
In all three cases not using the comparator (when available) can harm patients in the placebo group because they receive no treatment or treatment that is less than optimal. Examples of this bias include:
- the recent direct comparison of placebo with drugs developed for multiple sclerosis (e.g. cladribine, fingolimod, laquinimod)
- the combination of metformin with a new antidiabetic drug where placebo is used, despite established standard treatment combinations of two antidiabetic drugs
- the comparison of antidepressants where placebo is not necessary if the trial is designed to detect superiority.
Comparator
Selection of the comparator is very important because it can affect the evaluation of the new drug. Ideally the best standard should be used, at the best dose and duration of treatment. In practice these variables are often selected in order to favour the new drug. For instance in the case of rofecoxib, comparison with naproxen would have detected the cardiovascular adverse effects induced by rofecoxib. In another example, tacrolimus was shown to be superior to cyclosporin only because cyclosporin was used at suboptimal doses.
Non-inferiority trials
In this type of trial, investigators test the null hypothesis that a new drug is worse than the active control (standard therapy). When they can reject the null hypothesis, they accept the alternative, the new drug is not worse, but do we really need non-inferior drugs? Often the difference for acceptance of non-inferiority may be 25–50%. Patients rarely receive clear information in the informed consent form about the significance of this experimental design.
Surrogate end points
Frequently the evaluation of a drug is not based on therapeutic advantages for the patients, but on indicators that may indirectly reflect possible advantages. Examples are decreased blood cholesterol as a surrogate for reduction of myocardial infarction, decreased blood pressure as a surrogate for reduction of stroke, and decreased blood glucose as a surrogate for cardiovascular complications of diabetes. In some cases therapeutic end points are considered equivalent to surrogate end points, particularly when a group of drugs belong to a class with a similar mechanism of action.
However, since each drug has its own chemical structure, adverse reactions may in fact outweigh the benefit. Statins are an example. Rosuvastatin was approved for use on the basis of its hypocholesterolaemic effect, but simvastatin and pravastatin had already demonstrated protective effects against myocardial infarction. Cerivastatin was withdrawn because of toxicity. In another example, several anticancer drugs have been approved on the grounds that they reduce tumour volume, but with no evidence of improvement in overall survival or quality of life.
Composite end points
Where specific events are relatively few, it has become customary to group several events together. For instance death, myocardial infarction, stroke and coronary artery occlusion may be grouped as ‘cardiovascular events’. When a drug achieves a statistically significant decrease in a composite end point it is proclaimed that all the individual end points have been beneficially affected. However, the significance is usually driven by minor points that are less important therapeutically.
Fragile populations
New drugs are frequently tested on men rather than the population that will be using the drug, such as older people. Children are rarely recruited for trials, nor are women of fertile age. Furthermore, patients are selected for trials in artificial conditions, while in clinical practice patients may have multiple comorbidities and be taking several drugs already. Consequently, the trials may overestimate the drug’s benefits and underestimate its harms in clinical practice.
Publication bias
This term refers to the tendency to favour publications showing positive results rather than negative ones. This has created a number of problems in drug assessment. For instance, the selective serotonin reuptake inhibitors have been considered active in mild depression even though they are not different from placebo. Reboxetine is overall an ineffective and potentially harmful antidepressant when positive and negative trials are assessed together.3 The suicidal tendencies induced by antidepressant drugs in adolescents have not been well publicised.
Adverse reactions
The risk induced by drugs can seldom be detected during trials because they recruit too few people. Adequate postmarketing follow-up is therefore important. However, most drug withdrawals are instigated by pharmaceutical companies rather than the regulatory authority, indicating that appropriate follow-up by regulatory authorities does not occur. In addition, withdrawal is often considerably delayed, such as in the case of rofecoxib, cerivastatin, rosiglitazone, sibutramine, and rimonabant.