Automated Speech Analysis for Screening and Monitoring Bipolar Depression: Machine Learning Model Development and Interpretation Study.

BACKGROUND: Depressive episodes in bipolar disorder are frequent, prolonged, and contribute substantially to functional impairment and reduced quality of life. Therefore, early and objective detection of bipolar depression is critical for timely intervention and improved outcomes. Multimodal speech analyses hold promise for capturing psychomotor, cognitive, and affective changes associated with bipolar depression. OBJECTIVE: This study aims to develop between- and within-person classifiers to screen for bipolar depression and monitor longitudinal changes to detect depressive recurrence in patients with bipolar disorder. A secondary objective was to compare the predictive performance across speech modalities. METHODS: We collected 304 voice audio recordings obtained during semistructured interviews with 92 patients diagnosed with bipolar disorder over a 1-year period. Depression severity was assessed using the Hamilton Depression Rating Scale. Acoustic features were extracted using the openSMILE toolkit, and linguistic features were extracted using the Linguistic Inquiry and Word Count frameworks following automatic speech recognition and machine translation. Mixed-effects multivariate linear regression evaluated the associations between speech markers and Hamilton Depression Rating Scale scores adjusting for demographic variables, diagnosis, and feature-specific covariates. Extreme gradient boosting and light gradient boosting were used as base learners. We developed a between-person classifier to detect moderate to severe depression and a within-person classifier to detect recurrence. Hyperparameter tuning and 95% CI estimation were performed using a bootstrap bias-corrected cross-validation (k=5) approach combined with a grid search. Feature contributions were interpreted using Shapley additive explanations. RESULTS: Patients with depression showed reduced energy modulation, prolonged monotony, and more frequent use of words related to death and negative emotions. The between-person classifier combining acoustic and linguistic features detected moderate to severe depression with an area under the curve of 0.76 compared to 0.54 for the demographic model. The within-person classifier based on speech features detected depression recurrence with an area under the curve of 0.70 compared to 0.55 for the demographic model. CONCLUSIONS: Between- and within-person comparisons of speech markers can be leveraged in detecting and monitoring bipolar depression. We demonstrate the feasibility of applying Linguistic Inquiry and Word Count-based psycholinguistic analysis to machine-transcribed and translated speech, supporting the replicability of this approach across languages. Automated multimodal voice analysis can be integrated into digital health platforms, providing a scalable and effective approach for accessing mental health monitoring and care.