Randomised controlled clinical trials are the best source of evidence for assessing the efficacy of drugs. Observational studies provide critical descriptive data and information on long-term efficacy and safety that clinical trials cannot provide, at generally much less expense.
Observational studies include case reports and case series, ecological studies, cross-sectional studies, case-control studies and cohort studies. New and ongoing developments in data and analytical technology, such as data linkage and propensity score matching, offer a promising future for observational studies. However, no study design or statistical method can account for confounders and bias in the way that randomised controlled trials can.
Clinical registries are gaining importance as a method to monitor and improve the quality of care in Australia. Although registries are a form of cohort study, clinical trials can be incorporated into them to exploit the routine follow-up of patients to capture relevant outcomes.
Observational studies involve the study of participants without any forced change to their circumstances, that is, without any intervention.1 Although the participants’ behaviour may change under observation, the intent of observational studies is to investigate the ‘natural’ state of risk factors, diseases or outcomes. For drug therapy, a group of people taking the drug can be compared to people not taking the drug.
- Descriptive studies are primarily designed to describe the characteristics of a studied population.
- Analytical studies seek to address cause-and-effect questions.
Table: Summary of observational studies used in health research
|Case reports and case series||Descriptive||Easy to undertake
Can provide detailed information to assist hypothesis generation
|Relatively easy to
data can be used
|No data on
Profiling of a population or outcome of interest at a single time point5
|Relatively easy to undertake||Need for representative data|
Identify risk factors for a defined outcome (disease or condition)5,6
|Can be used to explore rare outcomes||Limited to a single outcome|
|Cohort studies||Descriptive and analytical
Estimate the incidence of outcomes of interest as well as their determinants5,7,8
Can be used to study multiple outcomes and multiple risk factors
|Relatively difficult and expensive|
Case reports and case series
Case reports and case series are strictly speaking not studies. However, they serve a useful role in describing new or notable events in detail. These events often warrant further formal investigation. Examples include reports of unexpected benefits or adverse events, such as a case report describing the use of high-dose quetiapine in treatment-resistant schizophrenia after intolerance to clozapine developed9 and a case report of a medication error involving lookalike packaging.10
Ecological studies are based on analysis of aggregated data at group levels (for example populations), and do not involve data on individuals. These data can be analysed descriptively, but not definitively for causation. Typical examples include studies that examine patterns of drug use over time. One example is the comparison of the use of non-steroidal anti-inflammatory drugs and COX-2 inhibitors in Australia and Canada.11 Sometimes ecological studies describe associations between drugs and outcomes, such as changes in the rates of upper gastrointestinal haemorrhage after the introduction of COX-2 inhibitors.12 However, because individual-level data are not presented, causality is at best only implied in ecological studies. The 'ecological fallacy' refers to the error of assuming that associations observed in ecological studies are causal when they are not.
Cross-sectional studies collect data at a single point in time for each single individual, but the actual data collection may take place over a period of time or on more than one occasion. There is no longitudinal follow-up of individuals. Cross-sectional studies represent the archetypal descriptive study.1 Typically, they provide a profile of a population of interest, which may be broad, like the Australian Health Survey undertaken intermittently by the Australian Bureau of Statistics,13 or focused on specific populations, such as older Australians.14
Case-control studies focus on determining risk factors for an outcome of interest (such as a disease or a drug’s adverse effect) that has already occurred.5
First, two groups of participants are assembled:
- those who already have the outcome (cases)
- those who do not have the outcome (controls), who are often matched to the cases to make them similar and reduce bias.
Second, data on previous exposure to selected risk factors are collected and compared to see if these risk factors are more (or less) common among cases versus controls. Case-control studies are useful for studying the risk factors of rare outcomes, as there is no need to wait for these to occur. Multiple risk factors can be studied, but each case-control study can involve only one outcome.5 One example explored the relationship between the use of antiplatelet and anticoagulant drugs (risk factor) and the risk of hospitalisation for bleeding (outcome) in older people with a history of stroke.15 Another case-control study explored the risk factors for the development of flucloxacillin-associated jaundice (outcome).16
Cohort studies compare outcomes between or among subgroups of participants defined on the basis of whether or not they are exposed to a particular risk or protective factor (defined as an exposure). They provide information on how these exposures are associated with changes in the risk of particular downstream outcomes. Compared to case-control studies, cohort studies take individuals with exposures and look for outcomes, rather than taking those with outcomes and looking for exposures. Cohort studies are longitudinal, that is they involve follow-up of a cohort of participants over time. This follow-up can be prospective or retrospective. Retrospective cohort studies are those for which follow-up has already occurred. They are typically used to estimate the incidence of outcomes of interest, including the adverse effects of drugs.
Cohort studies provide a higher level of evidence of causality than case-control studies because temporality (the explicit time relationship between exposures and outcomes) is preserved. They also have the advantage of not being limited to a single outcome of interest. Their main disadvantage, compared to case-control studies, has been that longitudinal data are more expensive and time-consuming to collect. However, with the availability of electronic data, it has become easier to collect longitudinal data.
One prospective cohort study explored the relationship between the continuous use of antipsychotic drugs (exposure) and mortality (outcome) and hospitalisation (outcome) in older people.17 In another older cohort, a retrospective study was used to explore the relationship between long-term treatment adherence (exposure) and hospital readmission (outcome).18
Observational studies versus randomised controlled trials
Compared to randomised controlled trials, observational studies are relatively quick, inexpensive and easy to undertake. Observational studies can be much larger than randomised controlled trials so they can explore a rare outcome. They can be undertaken when a randomised controlled trial would be unethical. However, observational studies cannot control for bias and confounding to the extent that clinical trials can. Randomisation in clinical trials remains the best way to control for confounding by ensuring that potential confounders (such as age, sex and comorbidities) are evenly matched between the groups being compared. In observational studies, adjustment for potential confounders can be undertaken, but only for a limited number of confounders, and only those that are known. Randomisation in clinical trials also minimises selection bias, while blinding (masking) controls for information bias. Hence, for questions regarding drug efficacy, randomised controlled trials provide the most robust evidence.
New and upcoming developments
New methods of analysis and advances in technology are changing the way observational studies are performed.
Clinical registries are essentially cohort studies, and are gaining importance as a method to monitor and improve the quality of care.19 These registries systematically collect a uniform longitudinal dataset to evaluate specific outcomes for a population that is identified by a specific disease, condition or exposure. This allows for the identification of variations in clinical practice20 and benchmarking across practitioners or institutions. These data can then be used to develop initiatives to improve evidence-based care and patient outcomes.21
An example of a clinical registry in Australia is the Australian Rheumatology Association Database,22 which collects data on the biologic disease-modifying antirheumatic drugs used for inflammatory arthritis. Clinical data from treating specialists are combined with patient-reported quality of life data and linked to national databases such as Medicare and the National Death Index. This registry has provided insight into the safety and efficacy of drugs and their effect on quality of life. It was used by the Pharmaceutical Benefits Advisory Committee to assess cost-effectiveness of these drugs.23
Another example is the Haemostasis Registry. It was used to determine the thromboembolic adverse effects of off-label use of recombinant factor VII.24
Clinical registries can also be used to undertake clinical trials which are nested within the registry architecture. Patients within a registry are randomised to interventions and comparators of interest. Their outcome data are then collected as part of the routine operation of the registry. The key advantages are convenience, reduced costs and greater representativeness of registry populations as opposed to those of traditional clinical trials.
One of the first registry-based trials was nested within the SWEDEHEART registry.25 This prospectively examined manual aspiration of thrombus at the time of percutaneous coronary intervention in over 7000 patients.26 The primary endpoint of all-cause mortality was ascertained through linkage to another Swedish registry. The cost of the trial was estimated to be US$400 000, which was a fraction of the many millions that a randomised controlled trial would have cost.
Propensity score matching
Even without randomising people within cohorts, methods have emerged in recent years that allow for less biased comparisons of two or more subgroups. Propensity score matching is a way to assemble two or more groups for comparison so that they appear like they had been randomised to an intervention or a comparator.27 In short, the method involves logistic regression analyses to determine the likelihood (propensity) of each person within a cohort being on the intervention, and then matching people who were on the intervention to those who were not on the basis of propensity scores. Outcomes are then compared between the groups. Propensity score analysis of a large cohort of patients with relapsing remitting multiple sclerosis found that natalizumab was superior to interferon beta and glatiramer acetate in terms of improved outcomes.28
Increasing sophistication in techniques for data collection will lead to ongoing improvements in the capacity to undertake observational studies (and also clinical trials). Data linkage already offers a convenient way to capture outcomes, including retrospectively. However, ethical considerations must be taken into account, such as the possibility that informed consent might be required before linking data. Machine learning will soon allow for easy analyses of unstructured text (such as free text entries in an electronic prescription).29 Patient-reported outcome measures are important and in future will be greatly facilitated by standardised, secure hardware and software platforms that allow for their capture, processing and analyses.
While clinical trials remain the best source of evidence regarding the efficacy of drugs, observational studies provide critical descriptive data. Observational studies can also provide information on long-term efficacy and safety that is usually lacking in clinical trials. New and ongoing developments in data and analytical technology offer a promising future for observational studies in pharmaceutical research.
Julia Gilmartin-Thomas is a Dementia research development fellow with the National Health and Medical Research Council (NHMRC) - Australian Research Council (ARC). Ingrid Hopper is supported by an NHMRC Early Career Fellowship.
Australian Prescriber welcomes Feedback
- Sedgwick P. Observational study designs. BMJ 2012;344:e4209.
- Carleton HA, Webb ML. The case report in context. Yale J Biol Med 2012;85:93-6.
- Nissen T, Wynn R. The clinical case report: a review of its merits and limitations. BMC Res Notes 2014;7:264.
- Sedgwick P. Ecological studies: advantages and disadvantages. BMJ 2014;348:g2979.
- Mann CJ. Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emerg Med J 2003;20:54-60.
- Sedgwick P. Case-control studies: advantages and disadvantages. BMJ 2014;348:f7707.
- Sedgwick P. Prospective cohort studies: advantages and disadvantages. BMJ 2013;347:f6726.
- Sedgwick P. Retrospective cohort studies: advantages and disadvantages. BMJ 2014;348:g1072.
- Chandrappa P, Ho L. Case reports of patients with treatment-resistant schizophrenia and related psychotic disorders intolerant to clozapine responding to high doses of quetiapine. Ther Adv Psychopharmacol 2012;2:207-9.
- Schnoor J, Rogalski C, Frontini R, Engelmann N, Heyde CE. Case report of a medication error by look-alike packaging: a classic surrogate marker of an unsafe system. Patient Saf Surg 2015;9:12.
- Barozzi N, Sketris I, Cooke C, Tett S. Comparison of nonsteroidal anti-inflammatory drugs and cyclooxygenase-2 (COX-2) inhibitors use in Australia and Nova Scotia (Canada). Br J Clin Pharmacol 2009;68:106-15.
- Mamdani M, Warren L, Kopp A, Paterson JM, Laupacis A, Bassett K, et al. Changes in rates of upper gastrointestinal hemorrhage after the introduction of cyclooxygenase-2 inhibitors in British Columbia and Ontario. CMAJ 2006;175:1535-8.
- Australian Bureau of Statistics. Australian Health Survey. 2016. [cited 2018 May 1]
- Morgan TK, Williamson M, Pirotta M, Stewart K, Myers SP, Barnes J. A national census of medicines use: a 24-hour snapshot of Australians aged 50 years and older. Med J Aust 2012;196:50-3.
- Quilliam BJ, Lapane KL, Eaton CB, Mor V. Effect of antiplatelet and anticoagulant agents on risk of hospitalization for bleeding among a population of elderly nursing home stroke survivors. Stroke 2001;32:2299-304.
- Fairley CK, McNeil JJ, Desmond P, Smallwood R, Young H, Forbes A, et al. Risk factors for development of flucloxacillin associated jaundice. BMJ 1993;306:233-5.
- Chan TC, Luk JK, Shea YF, Lau KH, Chan FH, Yu GK, et al. Continuous use of antipsychotics and its association with mortality and hospitalization in institutionalized Chinese older adults: an 18-month prospective cohort study. Int Psychogeriatr 2011;23:1640-8.
- Hochhalter AK, Basu R, Prasla K, Jo C. Retrospective cohort study of medication adherence and risk for 30-day hospital readmission in a Medicare Cost Plan. Manag Care 2014;23:43-7.
- Wilcox N, McNeil JJ. Clinical quality registries have the potential to drive improvements in the appropriateness of care. Med J Aust 2016;205:S21-6.
- Duggan A, Koff E, Marshall V. Clinical variation: why it matters. Med J Aust 2016;205:S3-4.
- Stey AM, Russell MM, Ko CY, Sacks GD, Dawes AJ, Gibbons MM. Clinical registries and quality measurement in surgery: a systematic review. Surgery 2015;157:381-95.
- Chen JS, Makovey J, Lassere M, Buchbinder R, March LM. Comparative effectiveness of anti-tumor necrosis factor drugs on health-related quality of life among patients with inflammatory arthritis. Arthritis Care Res (Hoboken) 2014;66:464-72.
- Reid CM. The role of clinical registries in monitoring drug safety and efficacy. Heart Lung Circ 2015;24:1049-52.
- Zatta A, McQuilten Z, Kandane-Rathnayake R, Isbister J, Dunkley S, Mcneil J, et al. The Australian and New Zealand haemostasis registry: ten years of data on off-licence use of recombinant activated factor VII. Blood Transfus 2015;13:86-99.
- Jernberg T, Attebring MF, Hambraeus K, Ivert T, James S, Jeppsson A, et al. The Swedish Web-system for enhancement and development of evidence-based care in heart disease evaluated according to recommended therapies (SWEDEHEART). Heart 2010;96:1617-21.
- Lagerqvist B, Fröbert O, Olivecrona GK, Gudnason T, Maeng M, Alström P, et al. Outcomes 1 year after thrombus aspiration for myocardial infarction. N Engl J Med 2014;371:1111-20.
- D’Agostino RB Jr. Propensity scores in cardiovascular research. Circulation 2007;115:2340-3.
- Spelman T, Kalincik T, Jokubaitis V, Zhang A, Pellegrini F, Wiendl H, et al. Comparative efficacy of first-line natalizumab vs IFN-β or glatiramer acetate in relapsing MS. Neurol Clin Pract 2016;6:102-15.
- Obermeyer Z, Emanuel EJ. Predicting the future - big data, machine learning, and clinical medicine. N Engl J Med 2016;375:1216-9.