Detecting Facial Action Units From Global-Local Fine-Grained Expressions
Date
2024
Authors
Zhang, W.
Li, L.
Ding, Y.
Chen, W.
Deng, Z.
Yu, X.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Journal article
Citation
IEEE transactions on circuits and systems for video technology (Print), 2024; 34(2):983-994
Statement of Responsibility
Wei Zhang, Lincheng Li, Yu Ding, Wei Chen, Zhigang Deng, Xin Yu
Conference Name
Abstract
Since Facial Action Unit (AU) annotations require domain expertise, common AU datasets only contain a limited number of subjects. As a result, a crucial challenge for AU detection is addressing identity overfitting. We find that AUs and facial expressions are highly associated, and existing facial expression datasets often contain a large number of identities. In this paper, we aim to utilize the expression datasets without AU labels to facilitate AU detection. Specifically, we develop a novel AU detection framework aided by the Global-Local facial Expressions Embedding, dubbed GLEE-Net. Our GLEE-Net consists of three branches to extract identity-independent expression features for AU detection. We introduce a global branch for modeling the overall facial expression while eliminating the impacts of identities. We also design a local branch focusing on specific local face regions. The combined output of global and local branches is firstly pre-trained on an expression dataset as an identity-independent expression embedding, and then finetuned on AU datasets. Therefore, we significantly alleviate the issue of limited identities. Furthermore, we introduce a 3D global branch that extracts expression coefficients through 3D face reconstruction to consolidate 2D AU descriptions. Finally, a Transformer-based multi-label classifier is employed to fuse all the representations for AU detection. Extensive experiments demonstrate that our method significantly outperforms the state-of-the-art on the widely-used DISFA, BP4D and BP4D+ datasets.
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
© 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.