Exploiting Multi-Emotion Relations at Feature and Label Levels for Emotion Tagging

Publication
MM ‘20: The 28th ACM International Conference on Multimedia, Virtual Event / Seattle, WA, USA, October 12-16, 2020

The dependence among emotions is crucial to boost emotion tagging. In this paper, we propose a novel emotion tagging method, that thoroughly explores emotion relations from both the feature and label levels. Specifically, a graph convolutional network is introduced to inject local dependence among emotions into the model at the feature level, while an adversarial learning strategy is applied to constrain the joint distribution of multiple emotions at the label level. In addition, a new balanced loss function that mitigates the adverse effects of intra-class and inter-class imbalance is introduced to deal with the imbalance of emotion labels. Experimental results on several benchmark databases demonstrate the superiority of the proposed method compared to state-of-the-art works.

Fig. The architecture of the proposed emotion tagging method. It consists of an emotional GCN Φg, an encoder Φe , a classifier Φc , and a discriminator D. dgcn denotes the product of the output of emotional GCN and the encoder feature, and d denotes the concatenation of dgcn and the output of the encoder. y and ŷ represent the real label and the predicted label, respectively.
Fig. The architecture of the proposed emotion tagging method. It consists of an emotional GCN Φg, an encoder Φe , a classifier Φc , and a discriminator D. dgcn denotes the product of the output of emotional GCN and the encoder feature, and d denotes the concatenation of dgcn and the output of the encoder. y and ŷ represent the real label and the predicted label, respectively.
Zhiwei Xu
Zhiwei Xu
Algorithm Engineer
Shangfei Wang
Shangfei Wang
Professor of Artificial Intelligence

My research interests include Pattern Recognition, Affective Computing, Probabilistic Graphical Models, Computation Intelligence.

Related