Deep learning-derived spatial organization features on histopathology images to predict prognosis in patients with colorectal liver metastasis, after hepatectomy.

person Xin Wang Department of Surgery, The Chinese University of Hong Kong, Hong Kong SAR, China info_outline Xin Wang, Yu-hong Li, Baocai Xing, Lin Qi, Jie-Ying Liang, Zhong-Wu Li, Shao-Yan Xi, Yu-Ni Lai, Feng Gao, De-Shen Wang, Ming-Tao Hu, Yi Cao, Xian-Rui Zhang, Li-Jian Xu, Ronald CK Chan

Authors person Xin Wang Department of Surgery, The Chinese University of Hong Kong, Hong Kong SAR, China info_outline Xin Wang, Yu-hong Li, Baocai Xing, Lin Qi, Jie-Ying Liang, Zhong-Wu Li, Shao-Yan Xi, Yu-Ni Lai, Feng Gao, De-Shen Wang, Ming-Tao Hu, Yi Cao, Xian-Rui Zhang, Li-Jian Xu, Ronald CK Chan Organizations Department of Surgery, The Chinese University of Hong Kong, Hong Kong SAR, China, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China, Peking University Cancer Hospital and Institute, Beijing, China, Department of Medical Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University, Guangzhou, China, Department of Pathology, Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education), Peking University Cancer Hospital & Institute, Beijing, China, Department of Pathology, Sun Yat-Sen University Cancer Center, Guangzhou, China, Department of Biomedical Sciences, City University of Hong Kong, Hong Kong SAR, China, Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases, The Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China, Centre for Perceptual and Interactive Intelligence, The Chinese University of Hong, Hong Kong SAR, China, Department of Pathology, The Chinese University of Hong Kong, Hong Kong SAR, China Abstract Disclosures Research Funding Other Foundation Science Technology and Innovation Commission of Shenzhen Municipality, Research Grants Council of the Hong Kong Special Administrative Region Background: Histopathological images of colorectal liver metastasis (CRLM) contain rich morphometric information that may predict patient outcomes, but current indicators depend on labor-intensive and subjective visual estimation. Herein, we aimed to develop an automated framework for tissue classification of routine H&E stained whole-slide images and establish a risk-scoring model for better prognosis prediction in patients with CRLM. Methods: Using 161,371 hand-delineated image patches, we trained a robust deep convolutional neural network- CRLM-SPA for accurate classification of CRLM into seven tissue types. With the tissue classification results on two independent in-house cohorts (SYSUCC: n = 433, BJCH: n = 404), we systematically quantified spatial organization features (SOFs), involving whole-slide, tumor-infiltrating and tumor-distal SOFs for different tissue types as well as the interactions between tumor and non-tumor tissues. Subsequently, univariate Cox proportional hazards regression analysis was performed to investigate the association between various SOFs and patient outcomes. Nonredundant SOFs that are clinically relevant were selected to build a risk-scoring model for prognosis using multivariate Cox regression analysis in the SYSUCC cohort, followed by validation in the BJCH cohort. Results: CRLM-SPA achieved an overall classification accuracy of > 93% in an independent set of 17,653 image patches. With the classification result, we calculated various SOFs and built a four-SOF risk-scoring model that significantly predicted overall survival (OS) in the discovery cohort, SYSUCC ( P = 1.40 10 -5 ; HR = 2.26; 95%CI: 1.55 - 3.29) and the independent validation cohort, BJCH ( P = 4.59 × 10 -4 ; HR = 2.00; 95%CI: 1.35 - 2.97). The prognostic performance of our SOF risk-scoring model is independent of the clinical risk score (CRS) system. Further stratification analyses in patients without preoperative chemotherapy (CTx) indicated that adjuvant CTx consistently improved OS in the SOF high-risk subgroups in both cohorts ( P = 0.018 and 0.049), but not in the SOF low-risk subgroups. Furthermore, a combined scoring system incorporating SOF and CRS considerably improved the prognostic performance in both cohorts over the individual SOF and CRS systems. Conclusions: CRLM-SPA showed a high accuracy in tissue classification and robustness in extracting prognostic information from H&E images. The SOF risk-scoring system demonstrated a strong and robust prognostic value that is independent of CRS, and could therefore provide a time- and cost-efficient tool to assist clinical decision making for patients with CRLM.