Synergistic face detection and pose estimation with energy-based models

Margarita Osadchy, Yann LeCun, Matthew L. Miller

Research output: Contribution to journalArticle

Abstract

We describe a novel method for simultaneously detecting faces and estimating their pose in real time. The method employs a convolutional network to map images of faces to points on a low-dimensional manifold parametrized by pose, and images of non-faces to points far away from that manifold. Given an image, detecting a face and estimating its pose is viewed as minimizing an energy function with respect to the face/non-face binary variable and the continuous pose parameters. The system is trained to minimize a loss function that drives correct combinations of labels and pose to be associated with lower energy values than incorrect ones. The system is designed to handle very large range of poses without retraining. The performance of the system was tested on three standard data sets - for frontal views, rotated faces, and profiles - is comparable to previous systems that are designed to handle a single one of these data sets. We show that a system trained simuiltaneously for detection and pose estimation is more accurate on both tasks than similar systems trained for each task separately.

Original languageEnglish (US)
Pages (from-to)1197-1215
Number of pages19
JournalJournal of Machine Learning Research
Volume8
StatePublished - May 2007

Fingerprint

Face Detection
Pose Estimation
Face recognition
Face
Energy
Labels
Model
Farthest Point
Binary Variables
Loss Function
Energy Function
Minimise
Range of data

Keywords

  • Convolutional networks
  • Energy based models
  • Face detection
  • Object recognition
  • Pose estimation

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Synergistic face detection and pose estimation with energy-based models. / Osadchy, Margarita; LeCun, Yann; Miller, Matthew L.

In: Journal of Machine Learning Research, Vol. 8, 05.2007, p. 1197-1215.

Research output: Contribution to journalArticle

@article{1df393a74b1341538dba42ee2635439b,
title = "Synergistic face detection and pose estimation with energy-based models",
abstract = "We describe a novel method for simultaneously detecting faces and estimating their pose in real time. The method employs a convolutional network to map images of faces to points on a low-dimensional manifold parametrized by pose, and images of non-faces to points far away from that manifold. Given an image, detecting a face and estimating its pose is viewed as minimizing an energy function with respect to the face/non-face binary variable and the continuous pose parameters. The system is trained to minimize a loss function that drives correct combinations of labels and pose to be associated with lower energy values than incorrect ones. The system is designed to handle very large range of poses without retraining. The performance of the system was tested on three standard data sets - for frontal views, rotated faces, and profiles - is comparable to previous systems that are designed to handle a single one of these data sets. We show that a system trained simuiltaneously for detection and pose estimation is more accurate on both tasks than similar systems trained for each task separately.",
keywords = "Convolutional networks, Energy based models, Face detection, Object recognition, Pose estimation",
author = "Margarita Osadchy and Yann LeCun and Miller, {Matthew L.}",
year = "2007",
month = "5",
language = "English (US)",
volume = "8",
pages = "1197--1215",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

TY - JOUR

T1 - Synergistic face detection and pose estimation with energy-based models

AU - Osadchy, Margarita

AU - LeCun, Yann

AU - Miller, Matthew L.

PY - 2007/5

Y1 - 2007/5

N2 - We describe a novel method for simultaneously detecting faces and estimating their pose in real time. The method employs a convolutional network to map images of faces to points on a low-dimensional manifold parametrized by pose, and images of non-faces to points far away from that manifold. Given an image, detecting a face and estimating its pose is viewed as minimizing an energy function with respect to the face/non-face binary variable and the continuous pose parameters. The system is trained to minimize a loss function that drives correct combinations of labels and pose to be associated with lower energy values than incorrect ones. The system is designed to handle very large range of poses without retraining. The performance of the system was tested on three standard data sets - for frontal views, rotated faces, and profiles - is comparable to previous systems that are designed to handle a single one of these data sets. We show that a system trained simuiltaneously for detection and pose estimation is more accurate on both tasks than similar systems trained for each task separately.

AB - We describe a novel method for simultaneously detecting faces and estimating their pose in real time. The method employs a convolutional network to map images of faces to points on a low-dimensional manifold parametrized by pose, and images of non-faces to points far away from that manifold. Given an image, detecting a face and estimating its pose is viewed as minimizing an energy function with respect to the face/non-face binary variable and the continuous pose parameters. The system is trained to minimize a loss function that drives correct combinations of labels and pose to be associated with lower energy values than incorrect ones. The system is designed to handle very large range of poses without retraining. The performance of the system was tested on three standard data sets - for frontal views, rotated faces, and profiles - is comparable to previous systems that are designed to handle a single one of these data sets. We show that a system trained simuiltaneously for detection and pose estimation is more accurate on both tasks than similar systems trained for each task separately.

KW - Convolutional networks

KW - Energy based models

KW - Face detection

KW - Object recognition

KW - Pose estimation

UR - http://www.scopus.com/inward/record.url?scp=34249661090&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34249661090&partnerID=8YFLogxK

M3 - Article

VL - 8

SP - 1197

EP - 1215

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -