Using large language models for clinical staging of colorectal cancer from imaging reports: a pilot study.

Purpose: Accurate data collection and analysis are crucial in clinical research, particularly for extracting information from unstructured medical records in cancer research. Traditional methods often struggle with this task. Large language models (LLMs) specializing in natural language processing (NLP), like ChatGPT (OpenAI), show potential for automating this process. This study evaluated whether GPT-4 could accurately extract clinical stages of colorectal cancer (CRC) from imaging reports. Methods: Using specific prompts based on the American Joint Committee on Cancer TNM staging system, GPT-4 was tested on the unstructured abdominal imaging reports of 100 CRC patients. The results were evaluated by a colorectal surgical oncologist and compared with data manually extracted by a nonspecialist data manager. Results: GPT-4 demonstrated high accuracy in extracting lesion locations (96.0%) and T (89.0%), N (90.0%), and M (85.0%) stages, with an overall TNM stage extraction accuracy of 69.0%. The combined accuracy for TNM stage and lesion location was 67.0%. Human data managers had similar TNM stage accuracy but lower lesion-location accuracy (76.0%). Higher accuracy was observed when reports directly mentioned stages and were in English only. Conclusion: This study confirms that LLM-based NLP, with proper prompt engineering, can accurately extract clinical stages from CRC imaging reports, particularly in English-only contexts.