A dual transfer learning method based on 3D-CNN and vision transformer for emotion recognition
Guardado en:
| Publicado en: | Applied Intelligence vol. 55, no. 2 (Jan 2025), p. 200 |
|---|---|
| Publicado: |
Springer Nature B.V.
|
| Materias: | |
| Acceso en línea: | Citation/Abstract Full Text - PDF |
| Etiquetas: |
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
MARC
| LEADER | 00000nab a2200000uu 4500 | ||
|---|---|---|---|
| 001 | 3147794579 | ||
| 003 | UK-CbPIL | ||
| 022 | |a 0924-669X | ||
| 022 | |a 1573-7497 | ||
| 024 | 7 | |a 10.1007/s10489-024-05976-z |2 doi | |
| 035 | |a 3147794579 | ||
| 045 | 2 | |b d20250101 |b d20250131 | |
| 084 | |a 137335 |2 nlm | ||
| 245 | 1 | |a A dual transfer learning method based on 3D-CNN and vision transformer for emotion recognition | |
| 260 | |b Springer Nature B.V. |c Jan 2025 | ||
| 513 | |a Journal Article | ||
| 520 | 3 | |a In the domain of medical science, emotion recognition based on electroencephalogram (EEG) has been widely used in emotion computing. Despite the prevalence of deep learning in EEG signals analysis, standard convolutional and recurrent neural networks fall short in effectively processing EEG data due to their inherent limitations in capturing global dependencies and addressing the non-linear and unstable characteristics of EEG signals. We propose a dual transfer learning method based on 3D Convolutional Neural Networks (3D-CNN) with a Vision Transformer (ViT) to enhance emotion recognition. This paper aims to utilize 3D-CNN effectively to capture the spatial characteristics of EEG signals and reduce data covariance, extracting shallow features. Additionally, ViT is incorporated to improve the model’s ability to capture long-range dependencies, facilitating deep feature extraction. The methodology involves a two-stage process: initially, the front end of a pre-trained 3D-CNN is employed as a shallow feature extractor to mitigate EEG data covariance and transformer biases, focusing on low-level feature detection. The subsequent stage utilizes ViT as a deep feature extractor, adept at modeling the global aspects of EEG signals and employing attention mechanisms for precise classification. We also present an innovative algorithm for data mapping in transfer learning, ensuring consistent feature representation across both spatio-temporal dimensions. This approach significantly improves global feature processing and long-range dependency detection, with the integration of color channels augmenting the model’s sensitivity to signal variations. In a 10-fold cross-validation experiment on the DEAP, experimental results demonstrate that the proposed method achieves classification accuracies of 92.44%<inline-graphic specific-use="web" mime-subtype="GIF" xlink:href="10489_2024_5976_Article_IEq1.gif" /> and 92.85%<inline-graphic specific-use="web" mime-subtype="GIF" xlink:href="10489_2024_5976_Article_IEq2.gif" /> for the valence and arousal dimensions, and the accuracies of four-class classification across valence and arousal are HVHA: 88.01%<inline-graphic specific-use="web" mime-subtype="GIF" xlink:href="10489_2024_5976_Article_IEq3.gif" />, HVLA: 88.27%<inline-graphic specific-use="web" mime-subtype="GIF" xlink:href="10489_2024_5976_Article_IEq4.gif" />, LVHA: 90.89%<inline-graphic specific-use="web" mime-subtype="GIF" xlink:href="10489_2024_5976_Article_IEq5.gif" />, LVLA: 78.84%<inline-graphic specific-use="web" mime-subtype="GIF" xlink:href="10489_2024_5976_Article_IEq6.gif" />. Similarly, it achieves an accuracy of 98.69%<inline-graphic specific-use="web" mime-subtype="GIF" xlink:href="10489_2024_5976_Article_IEq7.gif" /> on the SEED. Overall, this methodology not only holds substantial potential in advancing emotion recognition tasks but also contributes to the broader field of affective computing. | |
| 653 | |a Accuracy | ||
| 653 | |a Feature extraction | ||
| 653 | |a Medical science | ||
| 653 | |a Arousal | ||
| 653 | |a Classification | ||
| 653 | |a Affective computing | ||
| 653 | |a Emotion recognition | ||
| 653 | |a Artificial neural networks | ||
| 653 | |a Neural networks | ||
| 653 | |a Covariance | ||
| 653 | |a Recurrent neural networks | ||
| 653 | |a Algorithms | ||
| 653 | |a Electroencephalography | ||
| 653 | |a Emotions | ||
| 653 | |a Color sensitivity | ||
| 653 | |a Machine learning | ||
| 653 | |a Deep learning | ||
| 773 | 0 | |t Applied Intelligence |g vol. 55, no. 2 (Jan 2025), p. 200 | |
| 786 | 0 | |d ProQuest |t ABI/INFORM Global | |
| 856 | 4 | 1 | |3 Citation/Abstract |u https://www.proquest.com/docview/3147794579/abstract/embedded/75I98GEZK8WCJMPQ?source=fedsrch |
| 856 | 4 | 0 | |3 Full Text - PDF |u https://www.proquest.com/docview/3147794579/fulltextPDF/embedded/75I98GEZK8WCJMPQ?source=fedsrch |