NVIE

Overview

USTC-NVIE (Natural Visible and Infrared facial Expression) is a database for facial expression recognition and emotion inference. It was constructed by the Key Laboratory of Computing and Communication Software of Anhui Province (CCSL) with support from China’s National 863 Program. By combining spontaneous and posed expressions with visible and infrared thermal imagery, NVIE supports research on natural facial behavior under varying illumination conditions.

Dataset Description

NVIE contains facial expressions from more than 100 subjects, recorded simultaneously using a visible camera and an infrared thermal camera. Data were collected under three illumination conditions: front, left, and right. The posed database also includes images captured with and without glasses, allowing researchers to study the effects of glasses on expression recognition.

Dataset Composition

The database comprises two complementary subsets, each containing visible and infrared images:

  1. Spontaneous Expressions: Image sequences captured while subjects watched emotion-eliciting video clips, spanning expression onset to apex.
  2. Posed Expressions: Apex images of expressions performed on request, collected both with and without glasses.

Labels and Annotations

NVIE distinguishes between annotations of displayed facial expressions and subjects' self-reported emotions:

  • Expression Categories: Happiness, sadness, surprise, fear, anger, and disgust.
  • Observer Annotations: Five raters assessed spontaneous visible apex images for expression intensities, valence, and arousal on three-point scales. The dominant expression category and mean valence and arousal scores label the visible and infrared apex images.
  • Self-Reported Emotions: Subjects reported their experienced emotion category and rated its intensity, valence, and arousal on five-point scales, providing emotion annotations for the corresponding sequences.

Usage

NVIE can be used to train and evaluate facial expression recognition methods, compare spontaneous and posed expressions, and investigate emotion inference from facial thermal information. Its paired modalities and controlled recording conditions also support studies of illumination variation and glasses.

License and Agreement

NVIE is available for scientific research only. Commercial use and further redistribution are prohibited under the USTC-NVIE Database Release Agreement.

To request access, print and complete the agreement, obtain your supervisor’s signature and your department’s seal, and send the signed agreement to the contact below. Publications using NVIE must cite both papers listed below and forward a copy of the resulting publication or report to CCSL.

Citation

Please cite the database paper and the subsequent analysis paper when using NVIE:

@article{wang2010nvie,
  title={A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference},
  author={Wang, Shangfei and Liu, Zhilei and Lv, Siliang and Lv, Yanpeng and Wu, Guobing and Peng, Peng and Chen, Fei and Wang, Xufa},
  journal={IEEE Transactions on Multimedia},
  volume={12},
  number={7},
  pages={682--691},
  year={2010},
  doi={10.1109/TMM.2010.2060716},
  url={https://doi.org/10.1109/TMM.2010.2060716}
}

@article{wang2013nvie,
  title={Analyses of a Multimodal Spontaneous Facial Expression Database},
  author={Wang, Shangfei and Liu, Zhilei and Wang, Zhaoyu and Wu, Guobing and Shen, Peijia and He, Shan and Wang, Xufa},
  journal={IEEE Transactions on Affective Computing},
  volume={4},
  number={1},
  pages={34--46},
  year={2013},
  doi={10.1109/T-AFFC.2012.32},
  url={https://doi.org/10.1109/T-AFFC.2012.32}
}

Contact Us

For dataset access requests or any inquiries about NVIE, please contact:

Official Website of our Research Group: USTC-AC