1

In this paper, we propose employing the LRBN to effectively capture the high-order and global dependencies among facial geometric features

Most current works employ discriminative features and efficient classifiers for affective video content analyses, without explicitly exploring and leveraging domain knowledge for affective video content analyses. Therefore, in this paper, we propose a novel method to analyze affective video content through exploring domain knowledge. Both audio elements and visual elements are used by film makers to communicate emotions to audience. As a primary study to explore film grammar for affective video content analyses, this paper takes visual elements as an example to demonstrate the feasibility of the proposed affective video content analyses method enhanced through exploring domain knowledge.

Our contributions include the following aspects. First,we propose a regression approach for expression intensity estimation which exploits both ordinal relationship among different frames within an expression sequence and absolute intensity labels if available. Second, we introduce a unified max-margin learning framework to simultaneously exploit the two sources of information. An efficient algorithm to solve the optimization problem is developed. Third, our method can generalize to different learning settings depend-ing on the availability of expression intensity annotations.

Although their constructed representation reflects thermal infrared images’ supplementary role for visible images, it has no direct relationship to target expression labels. Furthermore, the hand-craft visible and thermal features may not thoroughly reflect the expression patterns embedded in images. Therefore, in this paper, we propose a new deep two-view approach to learn features from both visible and thermal images and leverage the commonality among visible and thermal images for expression recognition.