Shareable artificial intelligence to extract cancer outcomes from electronic health records.

person Kenneth L. Kehl Division of Population Sciences, Dana-Farber Cancer Institute and Harvard Medical School, Boston, MA info_outline Kenneth L. Kehl, Justin Jee, Karl Pichotta, Pavel Trukhanov, Christopher Fong, Michele Waters, Chelsea Nichols, Ethan Cerami, Deb Schrag, Nikolaus Schultz

Authors person Kenneth L. Kehl Division of Population Sciences, Dana-Farber Cancer Institute and Harvard Medical School, Boston, MA info_outline Kenneth L. Kehl, Justin Jee, Karl Pichotta, Pavel Trukhanov, Christopher Fong, Michele Waters, Chelsea Nichols, Ethan Cerami, Deb Schrag, Nikolaus Schultz Organizations Division of Population Sciences, Dana-Farber Cancer Institute and Harvard Medical School, Boston, MA, Memorial Sloan Kettering Cancer Center, New York, NY, Dana-Farber Cancer Institute, Boston, MA Abstract Disclosures Research Funding American Association for Cancer Research Doris Duke Charitable Foundation, National Cancer Institute Background: Clinical outcomes such as response, progression, and metastasis represent crucial data for observational cancer research, but outside of clinical trials, such outcomes are usually recorded only in unstructured notes in electronic health records (EHRs). Manual EHR annotation is too resource-intensive to scale to large datasets. Individual cancer centers have trained artificial intelligence natural language processing (AI/NLP) models to extract outcomes from their EHRs. However, due to concerns that models trained on protected health information (PHI) might encode private data, such models usually cannot be exported to other centers. Methods: EHR data from Dana-Farber Cancer Institute (DFCI) and Memorial-Sloan Kettering (MSK) collected through the AACR Project GENIE Biopharma Collaborative were used to train and evaluate Bidirectional Encoder Representations from Transformers (BERT)-based NLP models to extract outcomes from imaging reports and oncologist notes annotated with the Pathology, Radiology/Imaging, Signs/Symptoms, Medical oncologist, and bioMarkers (PRISSMM) framework. Document-level outcomes included the presence of active cancer; response; progression; and metastatic sites. ‘Teacher’ models trained on DFCI EHR data were used to label imaging reports and discharge summaries from the public MIMIC-IV dataset. ‘Student’ models trained to use MIMIC documents to predict teacher labels were transferred to MSK for evaluation. Results: Teacher models were trained at DFCI on 30,332 imaging reports for 2609 patients and 32,173 oncologist notes for 2917 patients with non-small cell lung, colorectal, breast, prostate, pancreatic, or urothelial cancer. The models were used to label 217,642 imaging reports and 141,377 discharge summaries from MIMIC for student model training. These DFCI-trained student models were evaluated at MSK, demonstrating high discrimination (AUROC > 0.90) across outcomes. Conclusions: This privacy-preserving “teacher-student” AI/NLP framework could expedite linkage of genomic data to clinical outcomes across institutions, accelerating precision cancer research. AUROCs of outcome extraction AI models trained at DFCI. Imaging Reports Oncologist Notes Model evaluation center DFCI DFCI MSK DFCI DFCI MSK Model type PHI-trained teacher model MIMIC- trained student model MIMIC-trained student model PHI-trained teacher model MIMIC- trained student model MIMIC-trained student model Outcome Any cancer 0.97 0.97 0.99 0.95 0.95 0.96 Progression 0.97 0.98 0.95 0.97 0.95 0.91 Response 0.96 0.95 0.96 0.96 0.95 0.94 Brain metastasis 0.99 0.99 0.99 Bone metastasis 0.99 0.99 0.99 Adrenal metastasis 0.99 0.99 0.99 Liver metastasis 0.99 0.99 0.99 Lung metastasis 0.98 0.97 0.98 Nodal metastasis 0.97 0.97 0.96 Peritoneal metastasis 0.99 0.99 0.97