From prediction to action: a retrospective observational study on the real-world implementation of Critical Interventions (CrIs), an AI-based clinical decision support system changing clinical behavior in the emergency department.

BACKGROUND: Optimal emergency department (ED) care for critically ill patients relies on prompt recognition and initiation of necessary interventions. The Critical Interventions (CrIs) system was designed to predict the need for urgent procedures, including arterial line insertion, oxygen therapy, high-flow nasal cannula (HFNC), intubation, and inotropes or vasopressors, based solely on triage data. This study aimed to evaluate the effectiveness of the CrIs system in enhancing clinical decision-making and reducing time to intervention, as well as to assess user satisfaction post-implementation. METHOD: A retrospective observational study was conducted at a tertiary hospital in South Korea, comparing time to interventions and outcomes between pre- and post-implementation phases of CrIs. The system's predictive performance was analyzed using the area under the receiver operating characteristic curve (AUROC). Additionally, user feedback was collected through surveys. RESULTS: The study included 37,632 ED visits, with notable predictive performance for CrIs across various interventions (AUROC: 0.879 for arterial lines, 0.906 for oxygen therapy, 0.95 for HFNC, 0.928 for intubation, and 0.88 for inotropes/vasopressors). Post-implementation, significant reductions were observed in the median time to intervention for A-lines, oxygen therapy, and HFNC. Survey responses highlighted overall satisfaction with CrIs, though suggestions for interface improvements were noted. CONCLUSION: The CrIs system demonstrated robust predictive accuracy and significantly reduced time to critical interventions, enhancing ED operational efficiency. Future enhancements should focus on improving user interface and communication features to maximize system utility. CLINICAL TRIAL NUMBER: Not applicable.