Face Detection, Pose Estimation and Landmark Localization in the Wild

Last updated on 02/10/2013



We present a unified model for face detection, pose estimation, and landmark estimation in real-world, cluttered images. Our model is based on a mixtures of trees with a shared pool of parts; we model every facial landmark as a part and use global mixtures to capture topological changes due to viewpoint. We show that tree-structured models are surprisingly effective at capturing global elastic deformation, while being easy to optimize unlike dense graph structures. We present extensive results on standard face benchmarks, as well as a new "in the wild" annotated dataset, that suggests our system advances the state-of-the-art, sometimes considerably, for all three tasks. Though our model is modestly trained with hundreds of faces, it compares favorably to commercial systems trained with billions of examples (such as Google Picasa and face.com).

X. Zhu, D. Ramanan. "Face detection, pose estimation and landmark localization in the wild" Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012.
[pdf]
[slides (Keynote file,36M)]
[slides (PPT file, converted from keynote, animation may not work correctly,17M)]


Updates

Movies

The following movies are DIVX encoded.

Lola Run Movie Lola Stop Movie

Downloads

FilenameDescriptionSize
README Description of contents. 2.3 KB
face-release1.0-basic.zip Basic code (matlab) for face detection, pose and landmark estimation with pre-trained models. 8.3 MB
face-release1.0-full.zip Full code (matlab) for training and testing. You need MultiPIE dataset to run it. 59 MB
sideview_annotation.zip[NEW] Landmark annotations of 400 sideview faces labeled by us. You also need the other annotations from CMU to reproduce the results in our paper (we don't have permission to release them here). 300 KB
mex-windows-compatible.zip Windows compatible mex files(.cc). 11 KB
models_CVPR2012.zip The fully shared and independent model we used to produce the curves in our CVPR2012 paper. 6.8 MB
AFW.zip[NEW] The Annotated Faces in the Wild (AFW) testset. 47 MB

FAQs