The analysis of real-world evidence (RWE) represents one of the avenues for the advancement of evidence-based medicine, clinical and economic analysis (CEA), and their application in health technology assessment (in the Russian Federation – comprehensive assessment of medicinal products). CEA is based on the systematic accounting of information on the clinical efficacy and safety of medical interventions, parameters of their utilization, impact on patients' quality of life, and certain other indicators. Currently, one of the primary sources of information to justify the inclusion of medicinal products in restrictive formularies is registrational clinical trials comparing the studied medical interventions with placebo. Over time, evidence has accumulated that the results of registrational clinical trials are not always reproducible in real-world clinical practice, leading to the recognition of the need to consider not only registrational trial data but also real-world data and evidence [1, 2].

RWE offers broad opportunities for application in the clinical evaluation of medicinal products when considering proposals for their inclusion in drug formularies. In clinical and economic studies (CES), RWE can also be used to select the CEA method, justify the choice of comparator drug, define the target patient population, as well as to provide transition probabilities in mathematical models and drug consumption parameters, and to model the patient pathway. In budget impact analysis (BIA), RWE can be used for the same purposes as in CES, as well as a source of information on the size and structure of the target patient population and the structure of current treatment practice [3].

Meanwhile, existing regulatory legal acts and methodological guidelines [4, 5] for conducting CEA for the purpose of including medicinal products in formularies do not contain explicit instructions on the possibilities of using RWE. Thus, it appears necessary to clarify the rules for using RWE in CEA [3, 6]. This thesis is also supported by the results of a survey of specialists involved in conducting comprehensive drug assessment, which showed that 55-69% of respondents support the consideration of RWE when analyzing the economic consequences of including medicinal products in formularies [7].

In the context of studying this direction, it seems appropriate to analyze the current practice of applying RWE in CEA. Examining the approaches used to utilize RWE, the structure of these approaches, areas of application, country of origin of data, sources, and certain other parameters could make it possible to systematize current practice. An analysis of domestic scientific literature indicates the limited nature of research in this area, which justifies the relevance of conducting this study.

Objective of this work was a retrospective analysis of the practice of using RWE in the comprehensive assessment of medicinal products during the preparation of dossiers, CES, and BIA for medicinal products proposed for inclusion in restrictive drug formularies.

Sources of RWE in this study included real-world studies and other real-world sources (statistical observations in healthcare, pharmacovigilance system data, mathematical models, etc.). Real-world studies in this study included any observational studies and pragmatic randomized controlled trials (RCTs). RWE studies also included studies based on the results of early/expanded/managed access programs, provided they had no restrictions on patient selection criteria for drug prescription leading to a narrowing of the target patient population compared to the medical use instructions, nor strict prescribing requirements.

To examine the use of RWE studies, a content analysis of dossiers prepared in accordance with legislative requirements to justify the inclusion of medicinal products in restrictive formularies in 2022 was conducted. The nature and purpose of using RWCP studies were determined by the context of their mention.

The practice of using RWE in CES and BIA was studied using published studies of medicinal products proposed for inclusion in drug formularies from 2018 to 2024. The search for published CES and BIA was conducted on the eLibrary.ru website, via the Yandex.ru and Google.com search engines, and within dossiers. Only CES conducted using mathematical modeling were included in the analysis. One source of RWE could be used to estimate several study variables; in such cases, each use was counted separately. The parameters of CES and BIA to which RWE related, the purpose of use, country of origin, and other attributes of the evidence were determined by the context of their application. The most frequently encountered applications of the evidence were grouped into the following categories: treatment outcomes (outcomes of treatment with the studied drugs or outcomes of treatment in second and subsequent lines after prescribing the studied drugs, other consequences of prescribing the studied drugs such as hospitalizations, frequency of diagnostic procedures, etc.), patient characteristics (size, socio-demographic characteristics, clinical characteristics including biomarker detection frequency and their quantitative parameters), structure of current treatment practice, health state utility, drug safety (frequency of adverse reactions or events during drug use), characteristics of medical care (justification of patient routing, list of medical services accounted for, frequency and multiplicity of their provision, duration, etc.), drug consumption parameters (dosage regimen, doses, duration of use, etc.). More specific applications were assigned to the "other" category.

Descriptive statistics were used in the study, calculating proportions, means, and medians. When medians are reported in the text of this article, the median value itself is provided, followed in parentheses by the interquartile range (first and third quartile values) and the range of values from minimum to maximum. When presenting proportions expressed as percentages, the numerator and denominator on which the value was calculated are provided in parentheses after them.

RWCP studies were used in 86% (24 out of 28) of dossiers. A total of 157 instances of using RWCP studies were identified. RWE studies were used to describe the features of disease treatment approaches (46% of all use instances), disease characteristics (32%), and disease epidemiology (22%) for which the proposed drug was intended.

According to the purpose of use, RWE was distributed as follows: justification of the need for the proposed drug, its advantages, and disadvantages of current approaches in 66% of use instances; determination of the target patient population size in 14%; use as a source of parameter values in clinical and economic calculations in 7%; determination of medical and demographic characteristics of the target patient population in 5%; description of the efficacy of the proposed drug in 4%; description of typical treatment practice in 1%; and in 3% of cases, the purpose of use was unclear.

By country of origin, foreign RWE studies were used in 61% of cases, Russian studies in 35%, and RWE studies obtained from an international cohort including Russian citizens were used in 4% of cases.

CES and BIA accounted for 29% of the instances of using RWE. More detailed statistics on the nature and use of RWCP studies in CES and BIA are presented in Table 1.

Table 1. Use of real-world studies in cost-effectiveness analysis and budget impact analysis as part of a dossier in 2022 (authors' data)

IndicatorsCEAsBIAsNumber of uses of real-world studies2126Of these:  non-local real-world studies18 (85.7%)18 (69.2%)local real-world studies3 (14.3%)8 (30.8%)Categories of parameters whose values were based on real-world studies  Epidemiology3 (14.3%)11 (42.3%)Characteristics of the disease10 (47.6%)5 (19.2%)Treatment patterns8 (38.1%)10 (38.5%)The purpose for using real-world studies  Source of model parameter values (except for calculating the target population size)9 (42.9%)-Justification of the need for therapy7 (33.3%)8 (30.8%)Description of the characteristics of the target population3 (14.3%)2 (7.7%)Calculating the target population size-15 (57.7%)Evaluation of the effectiveness of the proposed drug1 (4.8%)-Description of treatment patterns-1 (3.8%)Not clear1 (4.8%)-Abbreviations: CEA – cost-effectiveness analysis; BIA – budget impact analysis.

The search identified 106 CES, of which RWE was used in 43 CES. A total of 124 instances of using RWE were identified. By source, the evidence used came from RWE studies in 81% (101/124) of cases, from statistical healthcare observation data in 13% (16/124), from other sources in 4% (5/124), and in 2% of cases (2/124) the source was unclear. By country of origin, the majority of RWE uses were from foreign sources (62%). Regarding country of origin, the majority of uses of RWE from RWE studies were also based on foreign data, accounting for 71%.

The parameters of clinical and economic models most frequently estimated based on RWE included treatment outcomes (41%), patient characteristics (16%), structure of current treatment practice, and health state utility (both parameters at 7%). The main purposes of using RWE were more comprehensive cost accounting (34%), increasing the accuracy of cost calculations (24%), and accounting for the effectiveness of comparator drugs when calculating the cost-effectiveness ratio (19%).

In 16% (20/124) of use instances, RWE represented comparative data on the proposed drug and its comparator drug(s); in 6% (7/124) of instances, non-comparative data on the proposed drug; in 20% (24/124), non-comparative data on the comparator drug(s); in 56% (69/124), data not directly related to the compared drugs; and in 3% (4/124) of cases, the affiliation of the evidence was unclear. Summary data on the use of RWE in CES are presented in Table 2.

Table 2. Statistics on the use of RWE in CEA (authors' data)

CategoryOccurrence (N=124), absolute value (proportion)Model parameter estimated based on RWE Treatment outcomes and consequences51 (41.1%)Patient characteristics20 (16.1%)The structure of current treatment practice9 (7.3%)Utility of states9 (7.3%)Safety8 (6.5%)Characteristics of medical care8 (6.5%)Drug consumption parameters7 (5.5%)Not clear12 (9.7%)The purpose of using the RWE More comprehensive cost accounting42 (34.0%)Increase the accuracy of cost calculations30 (24.2%)Efficacy assessment for calculating cost-effectiveness ratio18 (14.5%)Selecting the CEA method and efficacy assessment for calculating cost-effectiveness ratio5 (4.0%)Calculating the target population size4 (3.2%)Selecting the CEA method3 (2.4%)Other purposes7 (5.6%)Not clear15 (12.1%)Abbreviations: CEA – cost-effectiveness analysis; RWE – real-world evidence.

RWE was used to select the CEA method in 14% (6/43) of CES. As a source of effectiveness data for calculating the cost-effectiveness ratio, RWE was used in 33% (14/43) of CES. In 16% (7/43) of CES, they characterized health state utility, which allowed the calculation of cost-effectiveness through quality of life adjustment. In all cases, utility values originated from foreign sources.

RWE was used to increase the accuracy of cost calculations in 51% (22/43) of CES and for more comprehensive cost accounting in 40% (17/43) of CES.

At the level of the entire identified sample of CES, including those in which RWE was not used, the choice of CEA method was justified using RWE in 6% (6/106) of cases. Effectiveness in RWE was considered when calculating the cost-effectiveness ratio in 13% (14/106) of CES, the accuracy of cost calculations was increased in 21% (22/106) of CES, and more comprehensive cost accounting was performed in 16% (17/106) of CES.

The search identified 82 BIAs, of which RWE was used in 52 studies. A total of 130 instances of using RWE were identified. Sources of evidence were RWCP studies in 65% (84/130) of cases, statistical healthcare observation data in 27% (35/130) of cases, other sources in 6% (8/130) of cases, and in 2% of cases (3/130) the source was unclear.

By country of origin, the majority of RWE uses were from domestic sources – 68%, foreign sources accounted for 31%, and in 1% it was unclear. Among RWE studies, the dominance of domestic sources diminished – 54% compared to 43%. All statistical healthcare observation data were Russian.

The parameters of clinical and economic models in BIA most frequently estimated based on RWE included patient characteristics (59%), treatment outcomes (19%), and structure of current treatment practice (9%). The main purposes of using RWE in BIA were calculating the target population size (51%, of which in 44% of cases the evidence was used as one of the multipliers in calculating the size, and in 7% it directly characterized the size of the target group), increasing the accuracy of cost calculations (22%), and more comprehensive cost accounting (18%).

In 9% (12/130) of use instances, RWE represented comparative data on the proposed drug and its comparator drug(s); in 6% (8/130) of instances, non-comparative data on the proposed drug; in 6% (8/130), non-comparative data on the comparator drug(s); in 78% (101/130), data not directly related to the compared drugs; and in 1% (1/130), the affiliation of the evidence was unclear. Summary data on the use of RWE in BIA are presented in Table 3.

Table 3. Statistics on the use of RWE in BIA (authors' data)

CategoryOccurrence (N=130), absolute value (proportion)Model parameter estimated based on RWE Patient characteristics77 (59.2%)Treatment outcomes and consequences24 (18.5%)The structure of current treatment practice11 (8.5%)Characteristics of medical care6 (4.6%)Drug consumption parameters6 (4.6%)Safety5 (3.8%)Not clear1 (0.8%)The purpose of using the RWE Calculating the target population size66 (50.8%)Increase the accuracy of cost calculations29 (22.3%)More comprehensive cost accounting23 (17.7%)Other purposes9 (6.9%)Not clear3 (2.3%)Abbreviations: BIA – budget impact analysis; RWE – real-world evidence.

The conducted analysis of approaches to using RWE in the evaluation of medicinal products to justify their inclusion in restrictive formularies revealed a wide diversity of purposes for their use and a significant number of indicators of the studied medicinal products, the target patient population, healthcare delivery practices, and long-term consequences of using the studied medicinal products, the values of which were determined based on the analysis of RWD.

In dossiers containing materials justifying the appropriateness of including medicinal products in restrictive formularies, RWE studies were used primarily to analyze current healthcare delivery practices, describe the disease for which the drug was proposed, its epidemiology, existing treatment approaches, limitations of current treatment methods, and features of the new drug. CES or BIA accounted for only one-third of all uses of RWE studies. The obtained result demonstrates the breadth of possibilities for using RWE in the comprehensive assessment of medicinal products, which indicates its significance in the formation of drug formularies.

In CES, RWE was most often used in cost calculation (58% of uses) and effectiveness assessment (choice of CEA method and/or calculation of the cost-effectiveness ratio, 21%). This result did not correspond to expectations. The main reason for interest in the field of RWD and evidence is generally considered to be the existence of discrepancies between the results of interventional clinical trials and RWE studies; based on this, it was expected that RWE would most often be used precisely in effectiveness assessment [1, 2, 8]. However, in the studied sample of CES, RWE was more often used in cost calculation rather than effectiveness. This observation suggests that, in practice, the relevance of RWE for CEA is primarily driven by the need for more comprehensive and accurate accounting of the economic consequences of using the studied drugs, and secondarily by the need to account for drug effectiveness in routine practice. From the perspective of interpreting the results of using RWE in CEA, it is now important to consider their possible impact not only on the magnitude of the effect of the compared drugs but also on the choice of comparator drug, the list of costs accounted for, the frequency and multiplicity of healthcare service provision, patient characteristics, etc.

In BIA, RWE was most often used to calculate the size of the target patient population, to increase the accuracy of cost calculations, and for more comprehensive cost accounting, which is entirely consistent with the features of this type of analysis, which does not require mandatory consideration of effectiveness data for the compared drugs.

The analysis of the use of RWE to assess effectiveness when calculating the cost-effectiveness ratio in CES revealed that data on the proposed drug and/or comparator drug(s) were used in only half of the cases. In the remaining half of cases, RWE characterized health state utility values or transition probabilities in mathematical models for states resulting from the use of second and subsequent lines of therapy. Moreover, across the entire sample of RWE uses in CES and BIA, most often they did not relate to the proposed drug and comparator drug(s) (56% in CES and 78% in BIA). Such RWE could characterize patient characteristics and size, efficacy, safety, and drug therapy consumption parameters for therapy prescribed after the use of the studied drug and comparator drug(s), health state utility, characteristics of medical services (list of services, frequency, multiplicity of provision, duration), and some other indicators. This identified feature also did not correspond to expectations, as the focus of discussions in the field of RWE research typically centers on new drugs (their effectiveness in real-world practice) or the effectiveness of comparator drugs in routine practice (necessary for use as an external control group when the new drug was studied in non-comparative clinical trials). The obtained result allows a new perspective on the role and nature of RWE – their use is not limited to or concentrated directly on the compared drugs, but allows characterizing and more fully accounting for the features of the other components of drug use practice, namely patients, as well as prior, concomitant, and long-term medical interventions (drugs and medical services). RWE allows not only to clarify the features and outcomes of using the compared drugs but also to provide a comprehensive and in-depth analysis of the consequences of widespread adoption of new drugs. The broad possibilities for using RWE in CEA are important for researchers conducting CES and BIA to consider in order to improve their practice.

The analysis of the country of origin of RWE indicates a general predominance of foreign sources. In 2022, fully foreign RWE studies were used in 61% of cases in drug dossiers. In CES within dossiers, their share increased to 86%, and in BIA – to 69%. In published CES, foreign RWE was used in 62% of cases, and among RWE studies – in 71% of cases. In BIA, conversely, Russian sources predominated, accounting for 68% of RWE uses due to the use of exclusively domestic sources in the category of statistical healthcare observations. Among RWE studies used in BIA, the share of domestic sources was closer to the share of foreign ones – 54% and 43%, respectively. The obtained data indicate the need for measures stimulating the generation and availability of Russian RWE. It is important to note separately that all sources of health state utility data were also foreign, which further underscores the need for conducting such studies in the Russian Federation.

The obtained results also allow for practical conclusions for future methodological guidelines on the use of RWE in the formation of restrictive drug formularies: it is advisable to consider the established practice of using RWE and to elaborate methodological issues of using RWE for a wide range of indicators, not only for assessing the efficacy and safety of the compared drugs.

RWE in the formation of restrictive drug formularies is used for a wide range of purposes and can describe a large number of indicators and parameters of the compared drugs, patients, and medical practice. RWE characterizes not only the proposed drug and its comparator drug(s) and most often does not relate to them but rather to the target patient population, features of healthcare delivery organization, or indirect (long-term) consequences of using the studied drugs. In CES, such evidence was most often used in cost calculation, less often in accounting for drug effectiveness. In BIA, RWE was most often used to calculate the size of the target patient population, less often in cost calculation and to expand the list of costs accounted for. Among the RWE used, foreign sources predominated, which highlights the need to develop measures aimed at supporting the generation of domestic RWE.