Privacy-Protected Facial Expression Recognition Augmented by High-Resolution Facial Images

Abstract

Cloud-based expression recognition from high-resolution facial images may put the subjects’ privacy at risk. We identify two kinds of privacy leakage, the appearance leakage in which the visual appearances of subjects are disclosed and the identity-pattern leakage in which the identity information of subjects is dug out. To address both leakages, we propose privacy-protected facial expression recognition from low-resolution facial images with the help of high-resolution facial images. Specifically, to prevent appearance leakage, we propose to extract identity-invariant representations from downsampled images, from which the visually distinguishable appearances cannot be recovered. To prevent identity-pattern leakage, we propose to eliminate the identity information from the extracted representations by leveraging the disentangled representations of high-resolution images as privileged information. After training, our method can fully capture identity-invariant representations from downsampled images for expression recognition without the requirement of high-resolution samples. These privacy-protected representations can be safely transmitted through the Internet. Experimental results in different scenarios demonstrate that the proposed method protects privacy without significantly inhibiting facial expression recognition.