Artificial Intelligence Use in Academic Applicant Screening: A Systematic Review.

Background: Escalating application volumes challenge holistic admissions review; artificial intelligence (AI) offers potential screening efficiencies. This systematic review examined AI algorithm use in academic admissions, categorizing types and evaluating objectives, performance, and ethical considerations. Methods: A systematic search of 5 databases through October 2024 identified original research on AI screening in undergraduate to medical residency programs. Data extraction covered study characteristics, AI types, performance metrics, and ethical considerations. Bias risk was assessed using the Newcastle-Ottawa Scale, and objective outcomes were compared statistically. Results: Eighteen studies (61,327 applicants) were included. The primary uses of AI were interview selection (n = 7), admissions decisions (n = 4), and reviewer scoring (n = 2). Algorithms included traditional machine learning (TML) and natural language processing (NLP). In the subset of studies reporting area under the receiver operating characteristic metrics, TML models averaged 0.89 versus 0.77 for NLP models. Ethical concerns, notably bias, were reported in 50% of studies. Conclusions: TML algorithms excelled with structured data; NLP showed value for unstructured data despite challenges (eg, transparency and data needs). Ethical concerns highlight the need for transparency and bias mitigation in AI adoption. Findings emphasize continued research to address ethical and methodological challenges.