Similarity Metric Learning and Applications to Unconstrained Face Recognition



Face recognition has attracted increasing attentions due to its applications in biometrics and surveillance. Recently, considerable research efforts are devoted to the unconstrained face verification problem, the task of which is to predict whether two face images represent the same person or not. The face images are taken under unconstrained conditions and show significant variations in complex background, lighting, pose, and expression (see e.g. Figure 1). In addition, the evaluation procedure for face verification typically assumes that the person identities in the training and test sets are exclusive, requiring the prediction of never-seen-before faces. Both factors make face verification very challenging, see e.g. Labeled Faces in the Wild (LFW) dataset. 




Figure 1: Example images from the LFW database exhibit large intra-personal variations: each column is a pair of images from the same person.



Similarity metric learning provides a natural solution for face verification by comparing image pairs based on the similarity or distance metric learnt from the data.   The basic intuition is that the similarity score between an image-pair from the distinct identity should be smaller than that between an image-pair from the same identity.  The critical challenges of this project are the following:


a)    How to extract visual features or template to represent face images?

b)   How to reduce noise and the large intra-personal variations?

c)    How to derive a visual similarity metric to effectively compare the similarity between face images?

d)   How to design elegant and efficient optimization algorithms for big face-image data?


Recently, we introduced in [1] a novel regularization framework of learning a similarity metric suitable for unconstrained face verification. We formulate its learning objective by incorporating the robustness to large intra-personal variations and the discrimination power of novel similarity metrics, a property most existing metric learning methods do not hold. Our formulation is a convex optimization problem which guarantees the existence of its global solution.



Table 1: Verification accuracy comparison on the restricted setting of LFW: our method [1] is denoted by Sub-SML.





Figure 2: ROC curve comparison on the restricted setting of LFW: our method [1] is Sub-SML.


 Our method Sub-SML proposed in [1] has achieved 89.73% on the restricted setting of the benchmark LFW database which is currently the best result reported so far, see Table 1 and Figure 2 for more comparison results.





1.    Y.Ying, Q. Cao and P. Li, Similarity metric learning for face recognition. IEEE International Conference on Computer Vision (ICCV), 2013. (First version, November 2012)

2.    Y. Ying and P. Li, Distance metric learning with eigenvalue optimization.   Journal of Machine Learning Research, 2012.

3.    Y. Ying, K. Huang and C. Campbell, Sparse metric learning via smooth optimization, Advances in Neural Information Processing Systems (NIPS), 2009.

4.    K. Huang, Y. Ying and C. Campbell, GSML: A unified framework for sparse metric learning, IEEE International Conference on Data Mining (ICDM), 2009.



Return to Yiming Ying's home page