Analytical methods in real-world data/evidence (RWD/E): a critical review

Introduction

The development and application of clinical evidence has resulted in a profound paradigm change in the healthcare environment [1]. Randomized controlled trials (RCTs) have long been regarded as the gold standard for assessing the efficacy and safety of medical products due to their ability to reduce bias through randomization [2]. However, RCTs frequently include highly selected patient populations, take place in controlled surroundings, and can be prohibitively expensive and time-consuming [3, 4]. These constraints frequently limit the generalizability of their findings to the numerous patient populations and clinical practice settings encountered in the real world [5, 6]. In response to these challenges, real-world data (RWD) has emerged as a crucial complementary source of information [7]. RWD refers to data about patient health status and/or healthcare delivery that is routinely obtained from a range of sources other than standard randomized controlled trials [8-10]. RWD is defined by the US Food and Drug Administration (FDA) as data produced from sources such as electronic health records (EHRs), medical claims data, product and disease registries, and data collected from mobile devices and other digital health technologies [11]. The examination of RWD yields real-world evidence (RWE), which is clinical evidence about the use and possible benefits or dangers of a medical product [11, 12]. This increased emphasis on RWD and RWE represents a fundamental shift in evidence creation, motivated by the awareness that RWD provides higher generalizability to various patient populations and real-world clinical settings, typically at a lower cost than traditional trials [13]. This trend suggests that regulatory agencies, payers, clinicians, and the pharmaceutical industry are increasingly valuing external validity and cost-efficiency alongside the internal validity previously prioritized by RCTs, necessitating a strong foundation for RWE creation [14, 15].

While RWD has enormous potential to provide a broader, more representative view of patient populations and clinical practice, its inherent complexities present significant analytical challenges [15]. Because RWD is not collected for research purposes, it leads to issues such as data quality inconsistencies, missing information, confounding factors, and various types of selection bias [12, 16]. These issues are common across various RWD sources and, if not addressed, can lead to misleading or inaccurate evidence. The principle "flawed input leads to flawed output" is especially important in the context of RWE [17]. Inadequate analytical approaches for RWD can produce unreliable data while also undermining trust in the broader RWD/RWE paradigm. If the data produced is insufficient, it can lead to bad decisions in crucial areas such as medication development, regulatory approvals, and clinical practice. Repeated instances of unreliable RWE may weaken trust among regulatory agencies, such as the FDA, and the broader medical community, hampering the very progress that RWD and RWE want to achieve. This circumstance highlights the important necessity for methodological transparency, thorough validation, and the use of sophisticated analytical approaches to ensure that the RWE generated is credible and actionable [18]. This review seeks to assess the range of analytical approaches used in RWD/RWE research, which include but not limited to statistical approaches, advanced machine learning algorithms, and specialized causal inference techniques to assess their respective strengths, limitations, and applicability in addressing the unique challenges of RWD/RWE analytics [8, 19]. Furthermore, it identifies present methodological limitations and suggests future options for developing and applying these analytical tools thereby aiming to be a significant resource for researchers, physicians, and policymakers seeking to understand and implement rigorous analytical approaches for producing high-quality and dependable RWE.

aiming to be a significant resource for researchers, physicians, and policymakers seeking to understand and implement rigorous analytical approaches for producing high-quality and dependable RWE.

Foundations of Real-World Data and Evidence

RWD includes information about patient health status and/or healthcare delivery that is frequently acquired from sources other than RCTs [5]; hence, understanding the fundamentals of RWD and RWE is critical for grasping their analytical journey from raw data to meaningful insights. This data comes from a variety of sources, including electronic health records (EHRs), medical claims data, product and disease registries, and, increasingly, data collected from mobile devices and other digital health technologies [20]. Raw data (RWD) is essentially the raw material (observations from routine clinical practice or administrative processes) for real-world evidence (RWE) [11]. RWE is the clinical evidence regarding the use and potential benefits or risks of a medical product derived from RWD analysis [12]; thus, it represents the refined product (the actionable insight that emerges after rigorous analytical processing of RWD [14]), and it is increasingly being used for critical functions such as regulatory decision-making, informing drug development strategies, and conducting post-market surveillance of medical products [5]. The quality and dependability of RWE are completely dependent on the quality of the underlying RWD and, more importantly, the analytical techniques used on it since such data maybe incomplete and lack end-point due to the fact that its initial purpose is not research [21].

Toward reproducibility, transparency, and ethical RWE for global healthcare impact

Fig. 1. From Real-World Data to Reliable Real-World Evidence

Common Sources and Types of RWD

RWD comes from a variety of sources, each providing unique insights and posing distinct analytical issues given than most of the collected data is unstructured, voluminous, and dynamic [22].

Electronic health records (EHRs) are comprehensive digital records of patient health information that include diagnoses, procedures, prescriptions, laboratory data, clinical notes, and more [18]. They provide extensive, longitudinal clinical detail, resulting in a thorough comprehension of patient journeys. However, because EHR data are primarily used for clinical treatment rather than research, they may contain structured and unstructured information (e.g. images, clinical notes etc.), missing data, coding variations, and errors [23, 24]. These require careful and intensive pre-processing. This type of RWD had created unprecedented opportunities for data-driven approaches in healthcare (pattern recognition, preoperative planning and others) [23, 25, 26]. This data has recently been reported to be more useful with the use of new analytical techniques such as Machine Learning and Neural networks to improve predictions in selected outcomes those linked to administrative and data usage hence allowing validation and replication of findings form RCT [27, 28].

Medical Claims Data are administrative datasets primarily generated for billing and reimbursement purposes, these provide a valuable insights into healthcare utilization and expenditures for large populations [29]. This type of data is a rich source for tracking diagnoses and procedures but has significant limitations. A notable drawback is the lack of detailed clinical information, such as specific laboratory results, disease severity, or lifestyle factors [26]. Furthermore, these datasets often only reflect services for which a claim was successfully processed, potentially excluding services that were not billed or reimbursed [23].

Patient registrations are organized systems that collect standardized data on individuals who have a given disease, condition, or are undergoing a specific intervention [26]. They frequently have greater data quality for specific outcomes and exposures important to their objective, but their generalizability may be limited to the group enrolled.

Patient-Generated Health Data (PGHD) is gathered through wearables and mobile technologies, is a rapidly expanding domain of real-world data [26]. These sources, which include data from wearable sensors and mobile applications, offer continuous and granular insights into patient behavior, physiological metrics (heart rate and sleep patterns), and symptoms in real time [30]. However, the use of PGHD presents notable challenges, including data heterogeneity, validity concerns, and significant privacy issues.

The integration of these diverse Real-World Data (RWD) sources such as Electronic Health Records (EHRs), claims data, patient registries, genomics, and PGHD results in complex, multi-modal datasets. While these datasets provide an unprecedented opportunity for a holistic understanding of patient health, they also amplify the complexities related to data harmonization, interoperability, and the selection of appropriate analytical methodologies [31].

Fig.2.RWD Types and Sources (source: [Swift et al., 2018])

Fig. 3. RWD sources and their respective data effectiveness

Challenges in RWD analytics

The utility of real-world data (RWD) is inherently dependent on both its quality and the robustness of the analytical methodologies employed. Since RWD is frequently collected for purposes unrelated to research, issues such as inconsistencies, coding variability, inaccurate entries, and substantial amounts of missing data are common, potentially undermining study validity [32]. Moreover, unlike randomized controlled trials (RCTs), the observational nature of RWD precludes, and random allocation of treatment lead to several potential biases. These include confounding by indication, in which treatment decisions are correlated with clinical outcomes; selection bias, arising from systematic differences between exposed and unexposed populations; and information bias, resulting from systematic errors in data collection [33, 34]. Heterogeneity in RWD is another key limitation, reflecting variability in patient demographics, clinical practice patterns, and data capture methodologies across healthcare systems, which complicates the synthesis and generalization of findings [35]. Furthermore, the non-experimental nature of RWD poses substantial challenges for causal inference, necessitating careful methodological adjustments to mitigate residual confounding and bias [36].

Table 1. Sources, data, application and limitations in RWD analytics

Analytical Methodologies for RWD/RWE

The unique characteristics of RWD necessitate a diverse toolkit of analytical methodologies, while foundational statistical methods remain relevant, the complexities of RWD often demand more advanced causal inference techniques and machine learning approaches to generate robust and reliable evidence [8].

Real-world evidence (RWE) studies frequently employ traditional statistical methods as foundational tools for descriptive analyses and preliminary hypothesis testing. Regression models, including linear regression for continuous outcomes, logistic regression for binary outcomes, and Cox proportional hazards regression for time-to-event outcomes, are commonly used to model associations and predict outcomes [22]. These models necessitate careful covariate adjustment to address confounding variables [37]. Propensity score methods are critical for mitigating confounding in observational studies by balancing observed covariates between treatment and control groups. Approaches include matching pairing treated and untreated individuals with similar propensity scores; weighting using inverse probability of treatment weighting (IPTW) to generate a balanced synthetic population; and stratification dividing participants into strata by propensity scores [38]. Causal inference methods aim to establish cause effect relationships in the presence of confounding and selection bias [18]. Instrumental variables (IV) exploit exogenous variation that influences treatment assignment but not the outcome, allowing estimation of causal effects even with unmeasured confounding. Difference-in-differences (DiD) compare pre- and post-intervention changes between treatment and control groups, while regression discontinuity designs (RDD) leverage threshold-based treatment assignment [12]. Target trial emulation explicitly structures observational studies to mimic randomized controlled trials, enhancing transparency in assumptions [39].

Beyond core analytical categories, advanced methodologies provide specialized capabilities for real-world data (RWD) analysis. Natural Language Processing (NLP) is particularly valuable for extracting structured information from unstructured text sources such as clinical notes, discharge summaries, and pathology reports within electronic health records (EHRs). By leveraging NLP, researchers can access nuanced clinical details often overlooked by structured data alone [40]. Bayesian methods offer a flexible statistical framework that incorporates prior knowledge or expert opinion into the analytical process. This approach is especially advantageous when addressing uncertainty or analyzing limited datasets, such as those in rare disease research [41]. Network analysis enables the exploration of complex interconnections among entities such as drugs, diseases, genes, or patients. Applying network theory to RWD can reveal important patterns, including drug-drug interactions, disease progression pathways, and patient similarity networks, thereby supporting a more integrated understanding of clinical and biological systems [42].

Table 2. Comprehensive Overview of Analytical Methods in Real-World Data Studies

Critical Appraisal of Analytical Methods

The generation of robust real-world evidence (RWE) depends on the careful analysis of real-world data (RWD) hence keen evaluation of analytical methods used must be done, particularly their strengths, limitations, and suitability for specific data and research contexts. By this, it allows us to understand the best methodology and also to predict the expected results.

Traditional statistical methods, such as regression models and propensity score techniques, remain foundational due to their interpretability and widespread use in controlling observed confounders [22, 37]. These approaches are well suited for hypothesis testing with structured data but are highly vulnerable to unmeasured confounding, restrictive model assumptions, and difficulties handling high-dimensional data [44, 45]. Causal inference methods, such as instrumental variables, regression discontinuity, and target trial emulation, explicitly aim to address confounding and selection bias [12, 18]. These approaches strengthen causal interpretation in observational settings and improve the credibility of findings for regulatory purposes [38, 46]. However, they rely on strong, often untestable assumptions such as valid instruments or no unmeasured confounding which make results sensitive to be specific and necessitate substantial methodological expertise [47].

Machine learning approaches excel at pattern discovery, prediction, and modeling non-linear interactions in high-dimensional RWD [43]. They automate feature extraction, support risk stratification, and enable novel insights into patient phenotypes. However, they are often criticized for their "black box" nature, raising interpretability and transparency challenges in clinical and regulatory contexts [43, 48]. Moreover, generalizability remains a concern, as models trained on one dataset may fail in another, underscoring the importance of rigorous validation [44]. Advanced methodologies such as natural language processing (NLP), Bayesian methods, and network analysis extend analytical capacity [40-42]. NLP enables extraction of rich information from unstructured clinical notes [49], Bayesian methods allow incorporation of prior knowledge to model uncertainty in sparse datasets [41], and network analysis reveals structural relationships among diseases, drugs, and patients [42]. Despite these strengths, their implementation requires specialized expertise, substantial computational resources, and careful handling of model assumptions [50].

Considerations for Bias, Confounding, and Generalizability

Bias mitigation and confounding control are critical in RWE research. While tools such as propensity scores and instrumental variables reduce confounding, no single method fully addresses unmeasured confounders [33, 35]. Sensitivity analysis and triangulation comparing results across multiple approaches therefore it is indispensable for robustness [22, 46]. Even although RWD generally enhances external validity compared to RCTs, machine learning models may still lack portability across populations, thus highlighting the necessity of external validation [51].

Table 3. Critical Appraisal of Analytical Methods in Real-World Evidence (RWE)

Applications and Future Directions of RWD/E

The analytical rigor applied to real-world data (RWD) translates into substantial impacts across multiple domains of healthcare, underscoring the transformative potential of real-world evidence (RWE).

Drug Development and Post-Market Surveillance. RWE is increasingly integrated into the lifecycle of medical products, from early development to post-market monitoring. It accelerates drug development by informing trial design, optimizing eligibility criteria, and identifying patient subgroups more likely to benefit from specific therapies [52]. Post-approval, RWE plays a crucial role in pharmacovigilance by detecting rare adverse events, evaluating long-term outcomes, and complementing randomized controlled trials (RCTs) with evidence from diverse populations in routine care [52, 53].

Comparative Effectiveness Research (CER). RWD supports comparative effectiveness research (CER) by enabling head-to-head comparisons of interventions in real-world clinical settings [12]. This extends beyond the controlled environment of RCTs to assess treatments as they are applied in practice, offering insights for treatment guidelines, payer decision-making, and policy development [54]. The observational design of CER makes it particularly valuable when RCTs are infeasible due to ethical or logistical constraints [6].

Health Economics and Outcomes Research (HEOR). HEOR relies heavily on RWE to evaluate the economic and societal value of interventions [55]. RWD enables cost-effectiveness assessments, burden-of-illness studies, and evaluations of real-world resource utilization, informing reimbursement decisions and value-based care models [55, 56].

Personalized Medicine and Precision Health. Personalized medicine depends on the integration of RWD with genomic, imaging, and phenotypic data to tailor treatments to individual characteristics [57]. These insights enable stratification of patients based on genetic or clinical markers, enhancing prediction of therapeutic responses and risks. The analytical shift is from population-level inference to individualized risk prediction, requiring advanced machine learning and statistical approaches to handle high-dimensional, multi-modal datasets [57, 58].

Emerging Analytical Techniques. Innovations in data science are critical for maximizing the value of RWD:

1. Federated Learning allows distributed model training across institutions without centralizing sensitive patient data, addressing privacy concerns and enabling large-scale, collaborative research [54].

2. Synthetic Data Generation uses generative AI to create realistic, privacy-preserving data for research and method development [55].

3. Explainable AI (XAI) seeks to improve interpretability of machine learning predictions, fostering trust in clinical practice and regulatory contexts [53].

Integration of Diverse Data Sources. Future progress depends on integrating heterogeneous RWD sources such as EHRs, claims, genomics, imaging, and social determinants of health into comprehensive patient profiles. Data harmonization and multi-modal analysis are necessary to derive robust insights [54].

Standardization and Reproducibility. A major challenge is the lack of standardization in data formats, terminologies, and analytic pipelines. Adoption of FAIR (Findable, Accessible, Interoperable, Reusable) principles and frameworks like the OMOP Common Data Model is essential to improve reproducibility and regulatory acceptance [53].

Role of AI and Advanced Computing. The scale and complexity of RWD necessitate the use of AI, high-performance computing, and cloud infrastructure for processing large datasets, training models, and conducting causal inference analyses. These tools are indispensable for unlocking the full potential of RWE [58].

Recommendations for Best Practices. Future directions emphasize methodological transparency, rigorous validation, interdisciplinary collaboration, and researcher training in advanced data science and ethics. These steps are crucial for ensuring the credibility, reproducibility, and ethical integrity of RWE [52].

Conclusion

Real-world data (RWD) and real-world evidence (RWE) have emerged as transformative pillars in contemporary healthcare, offering the ability to generate clinically meaningful insights that extend beyond the constraints of randomized controlled trials. By reflecting routine clinical practice, RWD enables the assessment of treatment effectiveness, safety, and patient-centered outcomes across diverse populations. However, the inherent complexities of RWD including but not limited to issues of data quality, missingness, heterogeneity, and confounding demand analytical rigor and innovation. This review has underscored the breadth of analytical methodologies available for RWD analysis, spanning traditional statistical models, causal inference frameworks, machine learning techniques, and advanced approaches such as natural language processing, Bayesian modeling, and network analysis. Each method presents distinct advantages and limitations, but no single strategy offers a comprehensive solution. A triangulated, multi-method approach that leverages complementary strengths is essential for enhancing validity, mitigating bias, and ensuring the robustness of findings. Future directions highlight the importance of emerging methods such as federated learning, synthetic data generation, and explainable artificial intelligence (XAI), which collectively address key concerns around data privacy, scalability, and interpretability. Equally critical are efforts to integrate heterogeneous data sources, promote standardization through common data models, and ensure reproducibility of findings across contexts. Methodological transparency, rigorous validation, and interdisciplinary collaboration will be indispensable in advancing the credibility and acceptance of RWE in clinical, regulatory, and policy decision-making. Ultimately, the promise of RWD and RWE lies not merely in methodological sophistication but in their ethical and responsible application. By aligning innovation with principles of fairness, privacy, and equity, the healthcare community can fully harness the potential of RWE to advance personalized medicine, inform evidence-based practice, and optimize healthcare delivery worldwide.