RWD & SYNTHETIC ARMS IN CLINICAL TRIALS
In contemporary medicine, every medical decision, (e.g. which type of treatment given to a patient), is founded on evidence-based medicine (EBM) [1]. The levels of evidence are organized through a ranking system to establish the strength of the results. These results are obtained through clinical trials or other types of research studies [2]. Obviously, the type of study design and the endpoints analyzed influence the strength of the evidence [2].
Randomized clinical trials (RCTs) are considered the best way to provide the highest levels of evidence in clinical practice since they allow to minimize possible sources of bias when testing new medical interventions. The reduction of selection or allocation bias is achieved through randomization of enrolled subjects in 2 or more cohorts receiving different interventions; this minimizes pre-clinical differences that could prejudge the outcome of the study. Blinding researchers also reduces other biases related to both researchers and subjects. The groups are then compared regarding a measured response. The experimental group receives the study drug, whereas the control group(s) receive an alternative intervention, a placebo, a substance without therapeutic value.
Even if RCTs are considered the gold standard for the conduction of clinical trials, the “ideal condition” to properly perform RCTs contains intrinsic limitations; the population studied in RCTs is for definition “selected” and thus is different from the real world care population [3], [4]. Furthermore, RCTs are run in a clinical trial setting, clearly different from the conventional scenario in clinical practice, making it difficult to explore complex interventions in compound populations [5], [6].
The use of placebo has been advantageous in RCT management since its first use in 1863, however in the last decades there have emerged several limitations. The first concern regards the placebo response, that is the difference between the result of no treatment and any change in the control group (the placebo effect) [7]. For this reason and increasing ethics concerns, nowadays, whenever it is feasible, clinical trials involve an active comparator, in other words, the new intervention should be measured against the current standard of care available and not just a placebo. This practice comes from the 1964 Declaration of Helsinki [8], which states that a new treatment should be verified against the best current therapeutic intervention Ethical analysis and international ethical guidance allow the use of placebo in RCTs in four cases: (1) when a confirmed effective treatment for the condition under study does not exist; (2) when avoiding the treatment produces small risks to participants; (3) when there are compelling methodological motives for using placebo and avoiding the treatment does not produce a significative risk to the participants; (4) when there are compelling methodological reasons for using placebo, and the research has the purpose to develop interventions that can be employed in the population from which the enrolled patients are drawn, and the trial does not necessitates participants to decline treatment they would otherwise receive [9].
Even if the use of an active comparator can improve ethical and logistics aspects related to the use of placebo, RCTs that employ an active control often face challenges and hurdles [10]. For example in a rapidly changing field such as oncology, the standard of care could become updated even during the development of the trial, interfering with the ethical foundation of the RCT and thus not justifying the randomization process [11]. On the other hand, in rare diseases, such as inherited conditions, it can be difficult to establish the active compactor, since often no effective treatments are available in these areas. Moreover, many clinical trials on rare diseases are carried out with few patients, leading to low statistical power, or are run as single-arm trials that do not permit to compare against other therapeutic options [12], [13].
With the advent and rise of precision medicine, overcoming these barriers and limitations has become a priority in clinical trial design. A possible solution consists of the use of external data sets with synthetic control methods. An external data is any source of clinical data from potentially relevant sources, which includes data from previous clinical trials, routinely collected health record information and patient registries (a collection of secondary data related to patients with a specific disease). Synthetic controls could thus be defined as cohorts of patients from external data set, adjusted using adequate statistical methodologies [10].
Both the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) have promoted several initiatives to use these new approaches in RCTs [14], [15].
The best way to produce a synthetic control arm
In the scenario of clinical trials, the use of external data to produce synthetic controls for clinical evaluations denotes a radical standard change. This is a new concept for clinical researches, and a certain degree of skepticism on their use is normal and somehow expected. It is however fundamental to expand among the scientific community the knowledge and advantages about the use of external data to create synthetic control groups. It is of great importance for researchers to be aware of the external data used and the statistical methods employed to create a synthetic control group [10].
The external data should be valid and reliable; for this reason, it is crucial to consider how the original collection of data was performed, compare the populations of the datasets compared, and the consistency and exhaustiveness of the datasets [10].
In case external control data are derived from similar RCTs and inclusion and exclusion criteria and patients’ features are comparable, the Bayesian methodology is a powerful tool to translate the external data into a prior distribution and combined with the concurrent RCT data [16]. This statistical approach permits a flexible degree of precision to be put on the external data [10]. On the contrary, in case the external control data are significantly different concerning enrollment criteria and patients’ characteristics, the simplest approach is to limit the external data to a subgroup of patients that match the concurrent clinical trial [10]. It is pivotal to restrict to the population, in regards to all parameters that are likely to cause any confounding (a variable that influences both the dependent variable and independent variable) [10]. After precise subgroups have been produced, external control data can be confronted with concurrent control data using approaches like the Bayesian one [10]. A “side effect” resulting from the restriction on several parameters can lead to the reduction of the sample size of the external data subgroups; in this case, it is better to use some form of statistical adjustment, rather than a reduction of the eligibility criteria of the external control data [10]. In this scenario the propensity score adjustment is a powerful statistical tool that brings different advantages; propensity score requires that all confounders are observed, thus it is easier to adapt a larger number of linear and non-linear relationships compared to multivariate regression, in case restricted sample sizes may represent a significant issue [17].
Finally, microsimulation is an alternative technique to explore long-term trends as synthetic control [10]; microsimulation is a form of modeling that refers to Markov models [18]. This approach leads to high-resolution definitions of the patient subgroup [10].
Main advantages related to synthetic arms in clinical trials
The development of the synthetic arm for clinical trials brought several benefits to the pharmaceutical industry and, more globally, to the clinical research field. Reducing or eliminating the enrollment of control groups, the use of synthetic control arm leads to the increase of efficiency, the decrease of delays, the reduction of trial costs, speeding up the marketing of lifesaving therapies. One of the reasons why patients chose to not participate in clinical trials lies in the concern of being randomized in the placebo arm; this fear is even bigger when the patient has a severe disease with a bad prognosis or the active comparator proposed has a known limited efficacy. The benefit linked to the use of a synthetic control arm guarantees that all participants receive the active treatment under study, removing important hurdles to recruitment.
In this social media era, patients could alter the blinding of the clinical trial by talking or writing about details of their participation in clinical studies trials, such as the treatment received and the developed side effects. The use of synthetic controls eliminates these concerns, keeping the integrity of the study.
During the SARS-CoV-2 pandemic, the use of synthetic control arm emerged as a crucial method to complete or restart trials disrupted by the ongoing outbreak.
Main limitations related to synthetic arms in clinical trials
Although synthetic control methods bring great opportunities, they are characterized by some limitations.
The main issues related to the use of synthetic control arms refer to the generalizability of the results and the needed statistical methodology. Furthermore, using a synthetic control arm, the disease under study should be predictable and its standard of care should be well-known and constant. Even if the required information is available from the real-world experience, sometimes they are of low quality or difficult to obtain, as the current electronic health records are typically isolated, fragmented, and difficult to access and share. In this scenario, new tools and techniques are required to better merge and organize real-world data guaranteeing that possible biases are minimized or eliminated.
Examples of studies that used synthetic arms in clinical trials
Davies and colleagues [19] compared the overall survival of anaplastic lymphoma kinase-positive non-small-cell lung cancer in patients who received alectinib with those who received ceritinib used two treatment arms, whose data were extracted from clinical trials and electronic health record database. The authors applied the propensity scores to balance baseline characteristics. Another example includes the procedure used by the European authorities to establish the pricing of alcensa, a monotherapy indicated for the first-line treatment of adult patients with anaplastic lymphoma kinase -positive advanced non-small-cell lung cancer. In this case, the authorities requested additional evidence of alecensa’s effectiveness relative to the standard of care ceritinib. The pharma company that produces alcensa, Roche, rather than waiting for the Phase 3 results, used a synthetic control arm of 67 patients to provide the necessary evidence of efficacy. This decision to use this methodology allowed the coverage of alecensa by 18 months in 20 European countries. A further example refers to the choice of pharma company Amgen to use a synthetic control arm to speed up the approval of blinatumomab for the treatment of a rare form of leukemia.
References
[1] Q.-V. Tran, “Studying a study and testing a test, how to read the medical evidence-fifth edition,” Ment. Heal. Clin., 2011, doi: 10.9740/mhc.n83630.
[2] N. C. I. at the N. I. of H. N. C. I. at the N. I. of Health., “NCI Dictionary of Cancer Terms: Levels of evidence. US DHHS-National Institutes of Health.” https://www.cancer.gov/publications/dictionaries/cancer-terms?CdrID=446533 (accessed Apr. 22, 2021).
[3] R. K. Albert, “‘Lies, damned lies...’ and observational studies in comparative effectiveness research,” American Journal of Respiratory and Critical Care Medicine. 2013, doi: 10.1164/rccm.201212-2187OE.
[4] J. H. Ware and M. B. Hamel, “Pragmatic Trials — Guides to Better Patient Care?,” N. Engl. J. Med., 2011, doi: 10.1056/nejmp1103502.
[5] B. K. Nallamothu, R. A. Hayward, and E. R. Bates, “Beyond the randomized clinical trial. The role of effectiveness studies in evaluating cardiovascular therapies,” Circulation. 2008, doi: 10.1161/CIRCULATIONAHA.107.703579.
[6] K. Stanley, “Design of randomized controlled trials,” Circulation, 2007, doi: 10.1161/CIRCULATIONAHA.105.594945.
[7] S. Chaplin, “The placebo response: an important part of treatment,” Prescriber, 2006, doi: 10.1002/psb.344.
[8] B. Shrestha and L. Dunn, “The Declaration of Helsinki on Medical Research involving Human Subjects: A Review of Seventh Revision,” J. Nepal Health Res. Counc., 2020, doi: 10.33314/jnhrc.v17i4.1042.
[9] J. Millum and C. Grady, “The ethics of placebo-controlled trials: Methodological justifications,” Contemp. Clin. Trials, 2013, doi: 10.1016/j.cct.2013.09.003.
[10] K. Thorlund, L. Dron, J. J. H. Park, and E. J. Mills, “Synthetic and external controls in clinical trials – A primer for researchers,” Clinical Epidemiology. 2020, doi: 10.2147/CLEP.S242097.
[11] S. P. Hey, A. J. London, C. Weijer, A. Rid, and F. Miller, “Is the concept of clinical equipoise still relevant to research?,” BMJ, 2017, doi: 10.1136/bmj.j5787.
[12] R. Joppi, V. Bertele’, and S. Garattini, “Orphan drugs, orphan diseases. The first decade of orphan drug legislation in the EU,” Eur. J. Clin. Pharmacol., 2013, doi: 10.1007/s00228-012-1423-2.
[13] F. J. Sasinowski, “Quantum of Effectiveness Evidence in FDA’s Approval of Orphan Drugs,” Drug Information Journal. 2012, doi: 10.1177/0092861511435906.
[14] F. and D. Administration, “Submitting Documents Using Real-World Data and Real-World Evidence to FDA for Drugs and Biologics Guidance for Industry;,” 2019.
[15] European Medicines Agency, “Guideline on Clinical Trials in Small Populations,” 2006.
[16] L. Dron, S. Golchi, G. Hsu, and K. Thorlund, “Minimizing control group allocation in randomized trials using dynamic borrowing of external control data – An application to second line therapy for non-small cell lung cancer,” Contemp. Clin. Trials Commun., 2019, doi: 10.1016/j.conctc.2019.100446.
[17] E. Williamson, R. Morley, A. Lucas, and J. Carpenter, “Propensity scores: From naïve enthusiasm to intuitive understanding,” 2012, doi: 10.1177/0962280210394483.
[18] U. Siebert et al., “State-transition modeling: A report of the ISPOR-SMDM modeling good research practices task force-3,” Value Heal., 2012, doi: 10.1016/j.jval.2012.06.014.
[19] J. Davies et al., “Comparative effectiveness from a single-arm trial and real-world data: Alectinib versus ceritinib,” J. Comp. Eff. Res., 2018, doi: 10.2217/cer-2018-0032.