An interpretable multimodal deep learning model for physiological stress detection.

Overview

1. Department of Computer Science, University of Mohamed Khider Biskra, Algeria
2. Department of Computer, Control and Management Engineering, Sapienza University of Rome, Italy
3. Laboratory of Telecommunications and Smart Systems, Faculty of Sciences and Technologies, University of Djelfa, Djelfa, Algeria
4. Department of Artificial Intelligence, ELTE Eotvos Lorand University, Budapest, Hungary
5. Division of Building Technology and Design, Department of Civil and Architectural Engineering, KTH Royal Institute of Technology, Stockholm, Sweden

Abstract

Stress is a major contributor to physical and mental health disorders, underscoring the need for reliable and interpretable detection systems. We present an interpretable multimodal deep learning framework that integrates electrocardiogram (ECG), electrodermal activity (EDA), and respiration signals using modality-specific neural architectures complemented by feature-based classical models. To ensure robustness and generalizability, the framework was evaluated under both subject-independent and subject-dependent protocols. Results show that multimodal fusion consistently outperformed single modalities, achieving 94.8% accuracy, 93.9% F1-score, and 0.97 AUC under subject-independent validation, compared with 86–91% accuracy for individual signals. In subject-dependent experiments, performance reached 97.2% accuracy and 96.8% F1-score. Comparative analysis with recent state-of-the-art methods demonstrated the competitiveness of our approach, while ablation studies confirmed the contribution of each modality and fusion strategy. Statistical significance testing with (p < 0.05) further validated the reliability of these gains. Importantly, interpretability analyses using Grad-CAM and attention visualization provided physiological insights, revealing that the model emphasizes clinically meaningful regions such as ECG R-peaks and phasic EDA fluctuations. By uniting high performance, robustness, and interpretability, this work advances stress detection research and offers a physiologically grounded framework for continuous mental health monitoring.

Reproduced under the paper's license (CC BY), from the paper cited above.

Code

The paper links to its data and to no code of its authors: see the Data section.

Tracing map

A tracing map links a paper to the code its authors published: this paper has none, so it has no map.

Data

Datasets cited

• figshare:31066015, at figshare; found in “Data Availability”

Data Availability

The data underlying the results presented in the study are available online from: https://figshare.com/articles/dataset/Processed_WESAD_Dataset/31066015.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 31 October 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 6 authors, 8 MeSH terms, 50 references.

Cite

This paper

Tibermacine, A., Tibermacine, I. E., Mancer, M., Rabehi, A., Guettala, W., & Habib, M. (2026). An interpretable multimodal deep learning model for physiological stress detection. PloS one, 21(10), e0360314. https://doi.org/10.1371/journal.pone.0360314

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1371/journal.pone.0335364

Conditional VAE for personalized neurofeedback in cognitive training.

Journal: PloS one

In common: 5 references

[2] doi:10.1038/s41598-026-50857-9

EEG-based stress classification using time-domain features and segmentation techniques.

Journal: Scientific reports

In common: methods / tools, 2 references

[3] doi:10.1371/journal.pone.0291070

ECG and EEG based detection and multilevel classification of stress using machine learning for specified genders: A preliminary study.

Journal: PloS one

In common: other, 2 references

[4] doi:10.1016/j.dib.2026.112822

Human experience in regulated offices (HERO) dataset: Synchronized multimodal physiological, environmental, perceptual, and video-derived measurements across structured office activities in a controlled indoor setting.

Journal: Data in brief

In common: other, methods / tools, 1 reference

[5] doi:10.3390/bios16080400

EEG and ECG Wearable Biosensor-Based Affective State Analysis Using Deep Learning over 6G IoT Healthcare Networks.

Journal: Biosensors

In common: other, 1 reference

[6] doi:10.1038/s41597-025-06328-3 [code]

EPIStress: A multimodal dataset of Physiological signals to measure cognitive stress in epilepsy patients.

Journal: Scientific data

In common: methods / tools, 1 reference

[7] doi:10.1523/eneuro.0104-22.2022 [code]

Enlarged Interior Built Environment Scale Modulates High-Frequency EEG Oscillations.

Journal: eNeuro

In common: other, 1 reference

[8] doi:10.1038/s41598-024-74475-5

EEG-based optimization of eye state classification using modified-BER metaheuristic algorithm.

Journal: Scientific reports

In common: methods / tools, 1 reference

[9] doi:10.3390/s25247634

EEG Sensor-Based Computational Model for Personality and Neurocognitive Health Analysis Under Social Stress.

Journal: Sensors (Basel, Switzerland)

In common: 1 reference

[10] doi:10.7717/peerj-cs.2649 [code]

Improved BCI calibration in multimodal emotion recognition using heterogeneous adversarial transfer learning.

Journal: PeerJ. Computer science

In common: 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.