Sampling methods for unsupervised learning

Rob Fergus, Andrew Zisserman, Pietro Perona

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present an algorithm to overcome the local maxima problem in estimating the parameters of mixture models. It combines existing approaches from both EM and a robust fitting algorithm, RANSAC, to give a data-driven stochastic learning scheme. Minimal subsets of data points, sufficient to constrain the parameters of the model, are drawn from proposal densities to discover new regions of high likelihood. The proposal densities are learnt using EM and bias the sampling toward promising solutions. The algorithm is computationally efficient, as well as effective at escaping from local maxima. We compare it with alternative methods, including EM and RANSAC, on both challenging synthetic data and the computer vision problem of alpha-matting.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004
PublisherNeural information processing systems foundation
ISBN (Print)0262195348, 9780262195348
StatePublished - 2005
Event18th Annual Conference on Neural Information Processing Systems, NIPS 2004 - Vancouver, BC, Canada
Duration: Dec 13 2004Dec 16 2004

Other

Other18th Annual Conference on Neural Information Processing Systems, NIPS 2004
CountryCanada
CityVancouver, BC
Period12/13/0412/16/04

Fingerprint

Unsupervised learning
Sampling
Computer vision

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Cite this

Fergus, R., Zisserman, A., & Perona, P. (2005). Sampling methods for unsupervised learning. In Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004 Neural information processing systems foundation.

Sampling methods for unsupervised learning. / Fergus, Rob; Zisserman, Andrew; Perona, Pietro.

Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004. Neural information processing systems foundation, 2005.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fergus, R, Zisserman, A & Perona, P 2005, Sampling methods for unsupervised learning. in Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004. Neural information processing systems foundation, 18th Annual Conference on Neural Information Processing Systems, NIPS 2004, Vancouver, BC, Canada, 12/13/04.
Fergus R, Zisserman A, Perona P. Sampling methods for unsupervised learning. In Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004. Neural information processing systems foundation. 2005
Fergus, Rob ; Zisserman, Andrew ; Perona, Pietro. / Sampling methods for unsupervised learning. Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004. Neural information processing systems foundation, 2005.
@inproceedings{c73cd15408504dd9b355739463196f08,
title = "Sampling methods for unsupervised learning",
abstract = "We present an algorithm to overcome the local maxima problem in estimating the parameters of mixture models. It combines existing approaches from both EM and a robust fitting algorithm, RANSAC, to give a data-driven stochastic learning scheme. Minimal subsets of data points, sufficient to constrain the parameters of the model, are drawn from proposal densities to discover new regions of high likelihood. The proposal densities are learnt using EM and bias the sampling toward promising solutions. The algorithm is computationally efficient, as well as effective at escaping from local maxima. We compare it with alternative methods, including EM and RANSAC, on both challenging synthetic data and the computer vision problem of alpha-matting.",
author = "Rob Fergus and Andrew Zisserman and Pietro Perona",
year = "2005",
language = "English (US)",
isbn = "0262195348",
booktitle = "Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004",
publisher = "Neural information processing systems foundation",

}

TY - GEN

T1 - Sampling methods for unsupervised learning

AU - Fergus, Rob

AU - Zisserman, Andrew

AU - Perona, Pietro

PY - 2005

Y1 - 2005

N2 - We present an algorithm to overcome the local maxima problem in estimating the parameters of mixture models. It combines existing approaches from both EM and a robust fitting algorithm, RANSAC, to give a data-driven stochastic learning scheme. Minimal subsets of data points, sufficient to constrain the parameters of the model, are drawn from proposal densities to discover new regions of high likelihood. The proposal densities are learnt using EM and bias the sampling toward promising solutions. The algorithm is computationally efficient, as well as effective at escaping from local maxima. We compare it with alternative methods, including EM and RANSAC, on both challenging synthetic data and the computer vision problem of alpha-matting.

AB - We present an algorithm to overcome the local maxima problem in estimating the parameters of mixture models. It combines existing approaches from both EM and a robust fitting algorithm, RANSAC, to give a data-driven stochastic learning scheme. Minimal subsets of data points, sufficient to constrain the parameters of the model, are drawn from proposal densities to discover new regions of high likelihood. The proposal densities are learnt using EM and bias the sampling toward promising solutions. The algorithm is computationally efficient, as well as effective at escaping from local maxima. We compare it with alternative methods, including EM and RANSAC, on both challenging synthetic data and the computer vision problem of alpha-matting.

UR - http://www.scopus.com/inward/record.url?scp=84898968638&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898968638&partnerID=8YFLogxK

M3 - Conference contribution

SN - 0262195348

SN - 9780262195348

BT - Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004

PB - Neural information processing systems foundation

ER -