PEPbench-Open, Reproducible, and Systematic Benchmarking of Automated Pre-Ejection Period Extraction Algorithms.

The pre-ejection period (PEP) is a widely used cardiac parameter in psychophysiology that reflects the duration between the onset of ventricular depolarization and the opening of the aortic valve. PEP is often used as a marker of cardiac sympathetic nervous system (SNS) activity, particularly in within-subject comparisons under similar hemodynamic conditions. While many algorithms for automated PEP extraction from electrocardiography (ECG) and impedance cardiography (ICG) signals (more precisely, its first derivative, dZ/dt) have been proposed in literature, they have not been systematically benchmarked. This lack of standardized algorithm comparisons originates from the absence of open-source algorithms and annotated datasets for evaluating PEP extraction algorithms. To address this issue, we introduce PEPbench, an open-source Python package with different Q-peak and B-point detection algorithms from literature that can be combined to create comprehensive PEP extraction pipelines, and a standardized framework for evaluating PEP extraction algorithms. We use PEPbench to systematically compare 108 different algorithm combinations. All combinations are evaluated on two datasets with manually annotated Q-peaks and B-points, which we make publicly available as the first datasets with reference PEP annotations. Our results show that the algorithms can differ vastly in their performance and that B-point detection algorithms introduce a considerable amount of error. Thus, we suggest that automated PEP extraction algorithms should be used with caution on a beat-to-beat level as their error rates are relatively high. This highlights the need for open and reproducible benchmarking frameworks for PEP extraction algorithms to improve the quality of research findings in the field of psychophysiology. With PEPbench, we aim to take a first step toward this goal and encourage other researchers to engage in the evaluation of PEP extraction algorithms by contributing algorithms, data, and annotations. We hope to establish a community-driven platform, fostering innovation and collaboration in the field of psychophysiology and beyond.