Supervised Machine Learning to Identify Hospital Inpatients Needing a Change of Antibiotic Therapy in Real Time: Preclinical Diagnostic Evaluation and Feasibility Study.

Background: Postprescription review (PPR) by clinical microbiology/infectious diseases specialists is a proven intervention for optimizing antibiotic management in hospitals. However, hospitals lack sufficient staff to conduct PPR at scale. This study investigated the feasibility and diagnostic performance of supervised machine learning to monitor electronic medical records and prioritize PPR in real time. Methods: Over 2 years, infection specialists categorized PPR recommendations as recommendations to "stop," "change," or "continue" therapy at an acute hospital in London, United Kingdom. These labels (n = 2625) were linked to features generated from electronic patient records. Random forest, XGBoost, and C5 classifiers were trained before undergoing prospective evaluation in an unseen representative validation dataset (446 PPR decisions). The prespecified minimum predictive performance target was an area under the curve (AUC) of 0.75, with statistical power to detect an AUC <0.68 or >0.82. We then aggregated the validation dataset at the patient level (n = 358) and compared the clinical utility of targeted PPR using the classifier against unprioritized (random) PPR. Results: In the prospective validation set, 145 of 358 patients (41%) were recommended to change or stop treatment. The best-fitting classifier (random forest, cross-validation AUC, 0.74) achieved an AUC of 0.70 (95% confidence interval, .65-.75). If just the top 30% of patients receiving antibiotics could be reviewed, the classifier would help stop or change treatment in 68 of 145 patients requiring a change, compared with 43 of 145 if patients were selected at random. Conclusions: The prospective clinical evaluation demonstrated that the approach is feasible and achieves moderate predictive performance in real-world conditions.