Face Detection, Pose Estimation and Landmark Localization in the Wild

Last updated on 02/10/2013

We present a unified model for face detection, pose estimation, and landmark estimation in real-world, cluttered images. Our model is based on a mixtures of trees with a shared pool of parts; we model every facial landmark as a part and use global mixtures to capture topological changes due to viewpoint. We show that tree-structured models are surprisingly effective at capturing global elastic deformation, while being easy to optimize unlike dense graph structures. We present extensive results on standard face benchmarks, as well as a new "in the wild" annotated dataset, that suggests our system advances the state-of-the-art, sometimes considerably, for all three tasks. Though our model is modestly trained with hundreds of faces, it compares favorably to commercial systems trained with billions of examples (such as Google Picasa and face.com).

X. Zhu, D. Ramanan. "Face detection, pose estimation and landmark localization in the wild" Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012.
[slides (Keynote file,36M)]
[slides (PPT file, converted from keynote, animation may not work correctly,17M)]



The following movies are DIVX encoded.

Lola Run Movie Lola Stop Movie


README Description of contents. 2.3 KB
face-release1.0-basic.zip Basic code (matlab) for face detection, pose and landmark estimation with pre-trained models. 8.3 MB
face-release1.0-full.zip Full code (matlab) for training and testing. You need MultiPIE dataset to run it. 59 MB
MultiPIE_annotations.zip Landmark annotations of multipie faces. 11 M
mex-windows-compatible.zip Windows compatible mex files(.cc). 11 KB
models_CVPR2012.zip The fully shared and independent model we used to produce the curves in our CVPR2012 paper. 6.8 MB
AFW.zip The Annotated Faces in the Wild (AFW) testset. 47 MB