TY - JOUR
T1 - Facial Action Units-Based Image Retrieval for Facial Expression Recognition
AU - Pham, Trinh Thi Doan
AU - Kim, Sesong
AU - Lu, Yucheng
AU - Jung, Seung Won
AU - Won, Chee Sun
N1 - Publisher Copyright:
© 2013 IEEE.
PY - 2019
Y1 - 2019
N2 - Facial expression recognition (FER) is a very challenging problem in computer vision. Although extensive research has been conducted to improve FER performance in recent years, there is still room for improvement. A common goal of FER is to classify a given face image into one of seven emotion categories: angry, disgust, fear, happy, neutral, sad, and surprise. In this paper, we propose to use a simple multi-layer perceptron (MLP) classifier that determines whether the current classification result is reliable or not. If the current classification result is determined as unreliable, we use the given face image as a query to search for similar images. In particular, facial action units are used to retrieve the images with a similar facial expression. Then, another MLP is trained to predict final emotion category by aggregating classification output vectors of the query image and its retrieved similar images. Experimental results on FER2013 dataset demonstrate that the performance of the state-of-the-art networks can be further improved by our proposed method.
AB - Facial expression recognition (FER) is a very challenging problem in computer vision. Although extensive research has been conducted to improve FER performance in recent years, there is still room for improvement. A common goal of FER is to classify a given face image into one of seven emotion categories: angry, disgust, fear, happy, neutral, sad, and surprise. In this paper, we propose to use a simple multi-layer perceptron (MLP) classifier that determines whether the current classification result is reliable or not. If the current classification result is determined as unreliable, we use the given face image as a query to search for similar images. In particular, facial action units are used to retrieve the images with a similar facial expression. Then, another MLP is trained to predict final emotion category by aggregating classification output vectors of the query image and its retrieved similar images. Experimental results on FER2013 dataset demonstrate that the performance of the state-of-the-art networks can be further improved by our proposed method.
KW - Convolutional neural networks
KW - facial action units
KW - facial expression recognition
KW - image retrieval
UR - http://www.scopus.com/inward/record.url?scp=85060533421&partnerID=8YFLogxK
U2 - 10.1109/ACCESS.2018.2889852
DO - 10.1109/ACCESS.2018.2889852
M3 - Article
AN - SCOPUS:85060533421
SN - 2169-3536
VL - 7
SP - 5200
EP - 5207
JO - IEEE Access
JF - IEEE Access
M1 - 8599142
ER -