Integrating a Shareable Artificial Intelligence Model Into Clinical Research for Cancer Recurrence in Patients With Breast and Colorectal Cancer.

PURPOSE: Cancer recurrence in clinical settings is documented in unstructured text, requiring labor-intensive manual record review to extract this outcome. A shareable natural language processing model developed at Dana-Farber Cancer Institute (DFCI)-DFCI-imaging-student-efficiently extracts cancer outcomes from radiology reports. We applied this model in a community oncology setting, aggregating report-level predictions to derive patient-level outcomes, and evaluated its performance in determining recurrence and time-to-recurrence in patients with breast cancer (BC) or colorectal cancer (CRC). METHODS: We randomly sampled 200 patients with BC and 200 patients with CRC from two cohorts at Kaiser Permanente Northern California. Patients were diagnosed with stage III disease (2005-2019) and followed until July 31, 2024, death, or disenrollment. We manually reviewed recurrence (local/regional/distant), recurrence date, and sites of recurrence using oncology, radiology, and pathology information in electronic health records. We then applied the DFCI-imaging-student model to radiology reports and compared recurrence based on the model outcomes against manual review. RESULTS: A total of 7,195 radiology reports were processed. During a median follow-up of 8.4 years for BC and 6.8 years for CRC, manual review identified 78 recurrence cases in BC (39%) and 70 in CRC (35%). The DFCI-imaging-student model demonstrated high sensitivity and specificity for recurrence detection in both cancers (breast: 92.3% and 92.6%, CRC: 94.3% and 86.9%) and moderate-to-high accuracy in identifying the sites of distant metastasis. Among true positives, the median error in time-to-recurrence was 0.16 months for breast and 0.48 months for CRC. CONCLUSION: Outcomes derived from the DFCI-imaging-student model output demonstrated high accuracy, providing an efficient determination of recurrence and time-to-recurrence in large-scale research to improve recurrence surveillance and facilitate collaborative research.