Synergistic face detection and pose estimation with energy-based models

Margarita Osadchy, Matthew L. Miller, Yann LeCun

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We describe a novel method for real-time, simultaneous multi-view face detection and facial pose estimation. The method employs a convolutional network to map face images to points on a manifold, parametrized by pose, and non-face images to points far from that manifold. This network is trained by optimizing a loss function of three variables: image, pose, and face/non-face label. We test the resulting system, in a single configuration, on three standard data sets - one for frontal pose, one for rotated faces, and one for profiles - And find that its performance on each set is comparable to previous multi-view face detectors that can only handle one form of pose variation. We also show experimentally that the system's accuracy on both face detection and pose estimation is improved by training for the two tasks together.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004
PublisherNeural information processing systems foundation
ISBN (Print)0262195348, 9780262195348
StatePublished - 2005
Event18th Annual Conference on Neural Information Processing Systems, NIPS 2004 - Vancouver, BC, Canada
Duration: Dec 13 2004Dec 16 2004

Other

Other18th Annual Conference on Neural Information Processing Systems, NIPS 2004
CountryCanada
CityVancouver, BC
Period12/13/0412/16/04

Fingerprint

Face recognition
Labels
Detectors

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Cite this

Osadchy, M., Miller, M. L., & LeCun, Y. (2005). Synergistic face detection and pose estimation with energy-based models. In Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004 Neural information processing systems foundation.

Synergistic face detection and pose estimation with energy-based models. / Osadchy, Margarita; Miller, Matthew L.; LeCun, Yann.

Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004. Neural information processing systems foundation, 2005.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Osadchy, M, Miller, ML & LeCun, Y 2005, Synergistic face detection and pose estimation with energy-based models. in Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004. Neural information processing systems foundation, 18th Annual Conference on Neural Information Processing Systems, NIPS 2004, Vancouver, BC, Canada, 12/13/04.
Osadchy M, Miller ML, LeCun Y. Synergistic face detection and pose estimation with energy-based models. In Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004. Neural information processing systems foundation. 2005
Osadchy, Margarita ; Miller, Matthew L. ; LeCun, Yann. / Synergistic face detection and pose estimation with energy-based models. Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004. Neural information processing systems foundation, 2005.
@inproceedings{97ebc575a852407482095209d81e31d6,
title = "Synergistic face detection and pose estimation with energy-based models",
abstract = "We describe a novel method for real-time, simultaneous multi-view face detection and facial pose estimation. The method employs a convolutional network to map face images to points on a manifold, parametrized by pose, and non-face images to points far from that manifold. This network is trained by optimizing a loss function of three variables: image, pose, and face/non-face label. We test the resulting system, in a single configuration, on three standard data sets - one for frontal pose, one for rotated faces, and one for profiles - And find that its performance on each set is comparable to previous multi-view face detectors that can only handle one form of pose variation. We also show experimentally that the system's accuracy on both face detection and pose estimation is improved by training for the two tasks together.",
author = "Margarita Osadchy and Miller, {Matthew L.} and Yann LeCun",
year = "2005",
language = "English (US)",
isbn = "0262195348",
booktitle = "Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004",
publisher = "Neural information processing systems foundation",

}

TY - GEN

T1 - Synergistic face detection and pose estimation with energy-based models

AU - Osadchy, Margarita

AU - Miller, Matthew L.

AU - LeCun, Yann

PY - 2005

Y1 - 2005

N2 - We describe a novel method for real-time, simultaneous multi-view face detection and facial pose estimation. The method employs a convolutional network to map face images to points on a manifold, parametrized by pose, and non-face images to points far from that manifold. This network is trained by optimizing a loss function of three variables: image, pose, and face/non-face label. We test the resulting system, in a single configuration, on three standard data sets - one for frontal pose, one for rotated faces, and one for profiles - And find that its performance on each set is comparable to previous multi-view face detectors that can only handle one form of pose variation. We also show experimentally that the system's accuracy on both face detection and pose estimation is improved by training for the two tasks together.

AB - We describe a novel method for real-time, simultaneous multi-view face detection and facial pose estimation. The method employs a convolutional network to map face images to points on a manifold, parametrized by pose, and non-face images to points far from that manifold. This network is trained by optimizing a loss function of three variables: image, pose, and face/non-face label. We test the resulting system, in a single configuration, on three standard data sets - one for frontal pose, one for rotated faces, and one for profiles - And find that its performance on each set is comparable to previous multi-view face detectors that can only handle one form of pose variation. We also show experimentally that the system's accuracy on both face detection and pose estimation is improved by training for the two tasks together.

UR - http://www.scopus.com/inward/record.url?scp=84898988145&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898988145&partnerID=8YFLogxK

M3 - Conference contribution

SN - 0262195348

SN - 9780262195348

BT - Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004

PB - Neural information processing systems foundation

ER -