Facial features, as one kind of important biometrics, can explicitly and implicitly represent the objective and subjective facial attributes (e.g., eyes, nose, mouth) and personal characteristics (e.g., identity, age, gender, races, emotion, beauty, personal character and hobbies). Learning facial features for detection and recognition of person identity, age, gender, races, expression, emotion, and beauty has been widely developed in computer vision and biometrics which has also greatly promoted the industrial applications of artificial intelligence. Currently, face recognition has been used in security inspection, access control system, video surveillance, etc. Additionally, age, emotion and beauty analysis have been used for multimedia, social and internet interaction. However, to the best of our knowledge, there is no research on exploration of a user′s psychological and emotional preference to different facial image styles towards recommendation applications.
Generally, it is very challenging to infer and reason about the implicit, fine-grained, subjective and common facial preference features that attract users from a very few selected face images of different styles (i.e., anchors) by the users. That is, if we could discover the facial preference features that the user internally and subjectively pays more attention to, then we can compute and predict the user′s personality preference via probabilistic models. After all, interesting applications with advanced emotional analysis, robot services, and automatic personalized image recommendation can be promoted by the discovered facial preference characteristic. It is worth noting that there have been a number of research works in facial beauty and attractiveness prediction, which, however, is essentially different from the proposed user specific preference inference and recommendation of different facial image styles in the following aspects.
1) Facial preference is less relevant to facial beauty that can be modeled with a universal criterion, while preference is user-specific and highly relevant to external facial styles (e.g., hairstyle, eye, nose, lips, glass).
2) Facial preference is also relevant to internal character reflected from faces (e.g., temperament, lovely, elegant), which is also user specific and even comprehensive for modeling users′ preference.
3) Due to the person-specific property of facial preference, the preference model parameters are dynamic and vary from person to person, while the facial beauty model is generally fixed and not person-specific. In other words, a face of highly beauty does not mean a high degree of a user′s preference, due to the users′ emotional difference.
Deep learning (DL), as a kind of supervised learning method originated from large-scale image recognition, has witnessed a huge success in multiple vertical fields, such as computer vision, pattern recognition, text analysis and speech recognition. Recently, transfer learning (TL), as a weakly-supervised cross-domain learning technique, has successfully promoted the horizontal development of DL in learning methodologies and applications. With the seamless connection between the supervised DL and the weakly-supervised TL, there is no doubt that DL and TL greatly stimulate the progress of artificial intelligence in many horizontal weakly-supervised research areas, such as medical image analysis, remote sensing image analysis, satellite image analysis, kinship verification, computer vision, load forecasting, fault diagnosis, etc. Generally, DL aims to obtain a universal and generalized knowledge representation model in a supervised manner, while TL aims to connect and propagate the DL knowledge to more weakly-supervised domains and tasks w/o fine-tune or partial re-training, where the data and labels are not completely or accurately prepared and deployed.
This paper is dedicated to the inference and reasoning analysis and modeling of a user′s psychological preference of facial image styles based on very few selected anchors (e.g., 10 images) by the user, which is undoubtedly a subjective, implicative, and weakly-supervised task. Therefore, a DL and TL inspired preference feature representation method is exploited for knowledge transfer from a large-scale supervised face recognition task to a single user-specific weakly-supervised face preference reasoning task. Further, probabilistic learning is used in the reasoning stage based on very few selected anchor faces (labeled as preferred faces by a user). Then, the model can be used to compute the psychological preference score for each gallery facial image, and the score value can successfully represent the degree of a user′s preference with respect to each gallery face.
The main contributions of this paper are four-fold.
1) It proposes an efficient DiscoStyle approach for user-specific facial preference reasoning and computation, by looking at very few anchors with more glances, which achieves automatic preference prediction and recommendation, which is the first work for users′ preference and face recommendation.
2) A deep transfer learning paradigm is proposed for facial preference related feature representation, based on a pre-trained face representation deep network, which comprehensively integrates the appearance features and geometric landmark feature for fully reflecting the facial style.
3) A multi-level logistic ranking (MLR) model with a novel on-line negative sample selection (ONSS) strategy is proposed in DiscoStyle for preference reasoning, which can predict the preference score and objectively define one user′s degree of preference to each gallery face and the faces of high preference degree are recommended.
4) A large facial style dataset (i.e., StyleFace) is developed for the first time for facial preference prediction, which includes a facial style subset for style attribute vector learning, an anchor subset for probabilistic reasoning and a gallery subset for preferred faces recommendation.
Download full text:
DiscoStyle: Multi-level Logistic Ranking for Personalized Image Style Preference Inference
Zhen-Wei He, Lei Zhang, Fang-Yi Liu
For more up-to-date information:
1) WeChat: IJAC
3) Facebook:International Journal of Automation and Computing
4) Linkedin: Int.J. of Automation and Computing
5) Sina Weibo:IJAC-国际自动化与计算杂志