Relevance. The relevance of the topic is determined by the fact that digital solutions that have demonstrated high accuracy on curated research datasets are increasingly being proposed for use in healthcare organizations, where data are shaped by the clinical practices of a particular institution.

Objective. To analyze factors related to real-world data (RWD) that may lead to a decline in the performance of medical artificial intelligence (AI) models when they are transferred to new settings, and to substantiate approaches to their preliminary assessment before implementation in a healthcare organization.

Materials and methods. A scoping review was conducted to systematize publications on the applicability of medical AI models beyond the original dataset. The search was performed in Scopus, Web of Science Core Collection, PubMed, Google Scholar and eLIBRARY. RU for the period from 2016 to 2026; earlier sources were included when necessary.

Results. The review identified the main types of medical data, causes of model performance degradation and approaches to assessing model applicability in a new clinical setting. Particular attention was paid to data shift as a cause of reduced accuracy, impaired calibration and an increased number of clinically significant errors. Approaches to external and local validation, assessment of model robustness in subgroups and post-implementation monitoring were considered. Practical requirements were also proposed for describing model applicability in a scientific publication or validation report.

Conclusion. The ability of a medical AI system to maintain its effectiveness when used outside the original environment should be assessed as a separate criterion of its reliability. Validation reports should provide a detailed and reproducible description of data sources, outcome definitions, testing conditions and results obtained in a specific healthcare organization, as well as a justified strategy for subsequent monitoring and maintenance of system performance.