The Ecological Assessment of Responses to Speaking-up tool-development and reliability testing of a method for coding safety listening behavior in naturalistic conversations.

Introduction: Safety communication is crucial for accident aversion across industries. While researchers often focus on encouraging concern-raising ('safety voice'), responses to these concerns ('safety listening') remain underexplored. Existing studies primarily use self-report measures; however, these tend to focus on perceptions of listening rather than behaviors. To fully understand and examine how safety listening is enacted and influential in safety-critical environments, a tool for reliably assessing naturalistic safety listening behaviors in high-risk settings is required. Accordingly, we developed and tested the Ecological Assessment of Responses to Speaking-up (EARS) tool to code safety listening behaviors in flightdeck conversations. Methods: There were three analysis phases: (1) developing the taxonomy through a qualitative content analysis (n = 45 transcripts); (2) evaluating interrater reliability and coder feedback (n = 40 transcripts); and (3) testing the taxonomy's interrater reliability in a larger unseen dataset (n = 110 transcripts) and with an additional coder (n = 50 transcripts). Results: Contrary to the notion that effective listening is agreement, our findings emphasize engagement with safety voice, including reasonable disagreement. The final taxonomy identifies six safety listening behaviors: action (implementing, declining), sensemaking (questioning, elaborating), and non-engagement (dismissing, token listening) and two additional voice acts (escalating, amplifying). EARS achieved substantial interrater reliability (Krippendorff's alpha of 0.73 to 0.77 and Gwet's ACT1 of 0.80 to 0.87). Discussion: The EARS tool allows researchers to assess safety listening in naturalistic conversations, facilitating analysis of its antecedents, its interplay with safety voice, and the impact of interventions on outcomes.