SERE Semi-Supervised Dual-Path Architecture for CLSER

Paper context

Paper title: Semantic-Emotional Resonance Embedding: A Semi-Supervised Paradigm for Cross-Lingual Speech Emotion Recognition Abstract: Cross-lingual Speech Emotion Recognition (CLSER) aims to identify emotional states in unseen languages. However, existing methods heavily rely on the semantic synchrony of complete labels and static feature stability, hindering low-resource languages from reaching high-resource performance. To address this, we propose a semi-supervised framework based on Semantic-Emotional Resonance Embedding (SERE), a cross-lingual dynamic feature paradigm that requires neither target language labels nor translation alignment. Specifically, SERE constructs an emotion-semantic structure using a small number of labeled samples. It learns human emotional experiences through an Instantaneous Resonance Field (IRF), enabling unlabeled samples to self-organize into this structure. This achieves semi-supervised semantic guidance and structural discovery. Additionally, we design a Triple-Resonance Interaction Chain (TRIC) loss to enable the model to reinforce the interaction and embedding capabilities between labeled and unlabeled samples during emotional highlights. Extensive experiments across multiple languages demonstrate the effectiveness of our method, requiring only 5-shot labeling in the source language. Passages referencing this figure: esonance is believed to stem from the mirror neuron system’s [ 2 ] mechanism of mapping others’ emotional experiences onto one’s own neural representations. However, current mainstream approaches heavily rely on large amounts of target-language emotion labels for supervised learning, which is difficult to achieve in low-resource language environments, thereby limiting their practical application. Figure 1: Traditional cross-lingual SER methods have significant drawbacks. Method A requires the presence of relevant emotional words for recognition, while B struggles with recognition due to the lack of explicit semantic content. Additionally, C relies on the assumption that the entire speech segment maintains a static emotional state. In contrast, our method D learns human emotional experience itional semi-supervised learning (SSL) [ 3 ] encompasses three paradigms: pseudo-labeling [ 12 ] , consistency regularization [ 19 ] , and graph propagation [ 7 ] . All rely on translation or alignment. In contrast, CLSER’s core lies in emotion-driven prosodic dynamic commonalities. These commonalities possess implicit features difficult to model through traditional alignment methods. As shown in Fig. 1 , to address the aforementioned challenges, our work proposes the SERE framework. Unlike traditional semi-supervised methods, SERE semantically anchors labeled data by learning human emotional experiences, while treating unlabeled data as a source for discovering cross-lingual affective resonance structures—rather than merely for label expansion—thus establishing a novel dynamic feature par