A MACHINE LEARNING MODEL FOR IDENTIFYING IMPORTANT CLINICAL VARIABLES AND PREDICTING ONE-YEAR TREATMENT RETENTION IN PATIENTS WITH RHEUMATOID ARTHRITIS PERFORMING AN INFLIXIMAB BIOSIMILAR-TO-BIOSIMILAR SWITCH

N. Prenga, H. Nabi, N. Steen Krogh, A. Akkoc, M. L. Hetland, B. GlintborgUniversity Hospital of Copenhagen Rigshospitalet, DANBIO and Copenhagen Center for Arthritis Research (COPECARE), Center for Rheumatology and Spine Diseases, Centre of Head and Orthopedics, Glostrup, Denmark University of Copenhagen, Department of Computer Science (DIKU), Copenhagen, Denmark ZiteLab ApS, Frederiksberg, Denmark Faculty of Health and Medical Sciences, University of Copenhagen, Department of Clinical Medicine, Copenhagen, Denmark IT University of Copenhagen, Copenhagen, Denmark University of Copenhagen, Department of Clinical Medicine, Faculty of Health and Medical Sciences, Copenhagen, Denmark  Background In Denmark, a nationwide mandatory infliximab biosimilar-to-biosimilar switch, from CT-P13 to GP1111, was recently conducted in patients with rheumatoid arthritis (RA) to save costs (ref). It is of interest to identify patients having favorable outcomes based on clinical characteristics at time of switch. Objectives We aimed to explore important baseline clinical factors by implementing a robust machine learning (ML) model to predict one-year GP1111 treatment retention. Methods Patients with RA who performed a biosimilar-to-biosimilar switch from CT-P13 to GP1111 between April 2019 -Feb 2020 (=switchers) were identified in the nationwide DANBIO registry. Clinical characteristics at GP1111 start (=baseline) were identified in DANBIO and through linkage to national patient registries (comorbidities). Missing baseline data were estimated using Multivariate Imputation by Chained Equations (20 imputations). A tree-based algorithm Extreme Gradient Boosting (XGBoost) was used to predict main outcome, defined as stop of GP1111 during one year of follow-up (stop/no stop). Contributing baseline characteristics were investigated as SHAP values. Due to imbalanced dataset (low stop rate), the ML model excellently predicted the majority outcome (no stop) but poorly the minority (stop). Thus, we incorporated rebalancing strategies (Random Oversampling, ROS, Random Undersampling, RUS, Synthetic Minority Oversampling Technique, SMOTE, Near Miss Algorithm, NearMiss) and the performance was inspected (confusion matrix). Optimal cut-offs (thresholds) were identified by Youden Index, Receiver Operating Characteristic Curve (ROC), Precision-Recall Curve to improve model-performance for the minority outcome. Analyses were performed in R and Python. Results We included 685 RA switchers whereof 17% stopped GP1111 during one year of follow-up (Table 1). SHAP values for 26 characteristics were investigated (Figure 1, see footnote). The ranges for accuracy and Area Under the ROC (AUC) were 79-87% and 0.78-0.85, with specificity and sensitivity ranging 0.79-0.99 and 0.01-0.12 respectively. This indicated a high true negative rate (specificity, predicting no stop of GP1111) and low true positive rate (sensitivity) i.e., poor prediction of GP1111 stop. With oversampling and undersampling methods, ranges for accuracy and AUC were 60-87% and 0.60-0.87 respectively, with specificity ranging 0.61-0.73 and sensitivity 0.50-0.67 corresponding to a higher true positive rate. This could be explained by RUS and NearMiss removing data from majority outcome and by ROS and SMOTE adding data to the minority outcome. Optimal threshold methods resulted in higher true positive rates and with a range for accuracy 70-85% (specificity 0.67-0.74, sensitivity 0.66-0.76). Table 1. Baseline characteristics in switchers. Stratified by stop/no stop of GP1111 during one year of follow-up No stop Stop Number of patients, n (%) 571 (83) 114 (17) Female, % 70 65 Age, years 47 (35, 56) 48 (37, 57) CDAI 4 (2,8) 8 (4,13) CRP, mg/L 2 (1, 5) 4 (1, 12) Concomitant methotrexate, % 78 61 Body mass index, kg/m2 25 (22, 29) 24 (22,27) HAQ 0.5 (0.1, 1.1) 1.0 (0.5, 1.5) Disease duration, years 11 (5, 20) 8 (5, 13) Physician global, VAS, mm 5 (2,10) 8 (2,20) Patient pain VAS, mm 22 (8, 43) 37 (22, 70) Numbers are median (IQR) unless otherwise stated Image/graph: Conclusion We implemented a robust machine learning model for predicting one-year retention in infliximab switchers and identified important baseline clinical characteristics. However, the model was challenged by imbalanced data (low stop rate) which was overcome by oversampling methods and optimal cut-offs. Careful methodological considerations are needed when advanced statistical models are applied to imbalanced datasets. Reference [1]Nabi et al, doi: 10.1136/rmdopen-2022-002560 Acknowledgements This project was supported by Vinnova, Innovationsfonden and The Research Council of Norway, under the frame of Nordforsk (Grant agreement no. 90825, Project NORA). Disclosure of Interests Nikolin Prenga: None declared, Hafsah Nabi Grant/research support from: AbbVie og Sandoz, Niels Steen Krogh: None declared, Ahmet Akkoc: None declared, Merete Lund Hetland Grant/research support from: AbbVie, Biogen, BMS, Celltrion, Eli Lilly, Janssen Biologics B.V., Lundbeck Foundation, Medac, MSD, Pfizer, Roche, Samsung Biopies, Sandoz, Novartis, Bente Glintborg Grant/research support from: Pfizer, AbbVie, BMS, Sandoz. Keywords: Prognostic factors, Artificial intelligence, Rheumatoid arthritis DOI: 10.1136/annrheumdis-2023-eular.2722Citation: , volume 82, supplement 1, year 2023, page 494Session: Rheumatoid arthritis - prognosis, predictors and outcome (Poster View)