Competency-Based Evaluation Methodology Enhances Learning During Internship: A Pilot Study.

Introduction: In light of the limitations of traditional internship evaluation methods, it is necessary to explore the application of a cpmpetency-oriented approach to internship assessment. The study aims to explore a new assessment method of internship learning that more meets the requirements of competence, and evaluate its impact on the effect of internship process in Department of Urology. Methods: The study participants comprised interns from the eight-year program at Peking Union Medical College (PUMC) who participated in the 1-month internship rotation in Department of Urology and participated in the after-department examination. The evaluation methodology was revised to implement a competency-oriented assessment utilizing standardized patients (SP). Participants were allocated into two groups: a control group unaware of the evaluation method prior to assessment (20 students), and an observation group informed of the updated evaluation methodology before internship (21 students). Both groups were evaluated and compared based on after-department examination performance, work engagement, and self-assessed ability improvement. The study lasted from March 1st, 2024 to October 31st, 2024. Results: The observation group demonstrated significantly higher scores than the control group (P<0.05) across the components of the after-department examination, including "decision-making", "communication and informed consent", "surgical detail", "postoperative management" and "total score". Similarly, the observation group achieved higher scores (P<0.05) in work engagement metrics, including "engagement in ward work", "engagement in communication", "engagement in surgery" and "total score". In self-assessed ability improvement, the observation group scored higher (P<0.05) in "surgical skill", "colleague cooperation ability", "communication ability" and "total score". Conclusion: The competency-oriented SP-based assessment method implemented in the Department of Urology internship demonstrates potential promotion for after-department examination performance, work engagement and self-evaluation of ability improvement. However, this conclusion is valid only for a single institution and department. Its application requires further evidence from higher-level studies involving multiple departments and centers.