高级检索

    多模态采集头盔在情感识别中的设计与研究

    Design and Research of Multimodal Acquisition Helmet in Emotion Recognition

    • 摘要: 情感识别在人类生活中起着重要的作用。文中设计了一款多模态采集头盔,通过硬件与算法的双重优化,全面提升情感识别的性能。首先结合脑电图(electroencephalogram,EEG)和眼动信号设计硬件采集系统,确保数据的高效与精确获取。然后设计了多模态对比学习门控网络(Multimodal Contrastive Learning Gated Network,MCGNet),以提高情感识别的准确率。针对EEG特征提取不全面的问题,设计了多域EEG特征提取器。为了更好地捕捉模态间的互补信息,模型使用对比表征学习和对比特征分解。最后,为解决多模态融合带来噪音干扰而导致准确率不如单模态的问题,引入门控结构,根据样本特征选择不同模态组合及相应专家网络。实验结果表明,该多模态采集头盔的性能良好,为情感识别的应用提供新思路。

       

      Abstract: Emotion recognition plays a crucial role in human life, which affects interpersonal communication and social behavior. A dual optimization of hardware and algorithms is presented in this paper to improve the performance of emotion recognition by using a multimodal acquisition helmet. The hardware acquisition system is designed to combine electroencephalogram (EEG) and eye movement signals to ensure efficient and precise data acquisition. The multimodal contrastive learning gated network (MCGNet) is developed to improve the accuracy of emotion recognition. Considering the insufficient extraction of EEG features, a multi-domain EEG feature extractor is designed. To better capture complementary information between modalities, the model uses contrastive representation learning and contrastive feature decomposition. To solve the problem of noise interference caused by multimodal fusion, which may lead to lower accuracy than single-modal methods, a gating structure is introduced to select different modality combinations and corresponding expert networks based on sample features. Experimental results demonstrate that the proposed multimodal acquisition helmet performs well, providing new insights into the application of emotion recognition.

       

    /

    返回文章
    返回