Affective Analyses

In this paper, we propose a new multimodal learning method, multimodal deep regression Bayesian network (MMDRBN), to construct the high-level joint representation of visual and audio modalities for emotion tagging. Then the MMDRBN is transformed into an inference network by minimizing the KL-divergence. After that, the inference network is used to predict discrete or continuous affective scores from video content.

Most current works employ discriminative features and efficient classifiers for affective video content analyses, without explicitly exploring and leveraging domain knowledge for affective video content analyses. Therefore, in this paper, we propose a novel method to analyze affective video content through exploring domain knowledge. Both audio elements and visual elements are used by film makers to communicate emotions to audience. As a primary study to explore film grammar for affective video content analyses, this paper takes visual elements as an example to demonstrate the feasibility of the proposed affective video content analyses method enhanced through exploring domain knowledge.