Loss functions for discriminative training of energy-based models

Yann LeCun, Fu Jie Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Probabilistic graphical models associate a probability to each configuration of the relevant variables. Energy-based models (EBM) associate an energy to those configurations, eliminating the need for proper normalization of probability distributions. Making a decision (an inference) with an EBM consists in comparing the energies associated with various configurations of the variable to be predicted, and choosing the one with the smallest energy. Such systems must be trained discriminatively to associate low energies to the desired configurations and higher energies to un-desired configurations. A wide variety of loss function can be used for this purpose. We give sufficient conditions that a loss function should satisfy so that its minimization will cause the system to approach to desired behavior. We give many specific examples of suitable loss functions, and show an application to object recognition in images.

Original languageEnglish (US)
Title of host publicationAISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics
Pages206-213
Number of pages8
StatePublished - 2005
Event10th International Workshop on Artificial Intelligence and Statistics, AISTATS 2005 - Hastings, Christ Church, Barbados
Duration: Jan 6 2005Jan 8 2005

Other

Other10th International Workshop on Artificial Intelligence and Statistics, AISTATS 2005
CountryBarbados
CityHastings, Christ Church
Period1/6/051/8/05

Fingerprint

Discriminative Training
Loss Function
Configuration
Energy
Object recognition
Probability distributions
Model
Object Recognition
Graphical Models
Probabilistic Model
Normalization
High Energy
Probability Distribution
Sufficient Conditions

ASJC Scopus subject areas

  • Artificial Intelligence
  • Statistics and Probability

Cite this

LeCun, Y., & Huang, F. J. (2005). Loss functions for discriminative training of energy-based models. In AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (pp. 206-213)

Loss functions for discriminative training of energy-based models. / LeCun, Yann; Huang, Fu Jie.

AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics. 2005. p. 206-213.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

LeCun, Y & Huang, FJ 2005, Loss functions for discriminative training of energy-based models. in AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics. pp. 206-213, 10th International Workshop on Artificial Intelligence and Statistics, AISTATS 2005, Hastings, Christ Church, Barbados, 1/6/05.
LeCun Y, Huang FJ. Loss functions for discriminative training of energy-based models. In AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics. 2005. p. 206-213
LeCun, Yann ; Huang, Fu Jie. / Loss functions for discriminative training of energy-based models. AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics. 2005. pp. 206-213
@inproceedings{1edfe574951c45108c38dcb62539def1,
title = "Loss functions for discriminative training of energy-based models",
abstract = "Probabilistic graphical models associate a probability to each configuration of the relevant variables. Energy-based models (EBM) associate an energy to those configurations, eliminating the need for proper normalization of probability distributions. Making a decision (an inference) with an EBM consists in comparing the energies associated with various configurations of the variable to be predicted, and choosing the one with the smallest energy. Such systems must be trained discriminatively to associate low energies to the desired configurations and higher energies to un-desired configurations. A wide variety of loss function can be used for this purpose. We give sufficient conditions that a loss function should satisfy so that its minimization will cause the system to approach to desired behavior. We give many specific examples of suitable loss functions, and show an application to object recognition in images.",
author = "Yann LeCun and Huang, {Fu Jie}",
year = "2005",
language = "English (US)",
isbn = "097273581X",
pages = "206--213",
booktitle = "AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics",

}

TY - GEN

T1 - Loss functions for discriminative training of energy-based models

AU - LeCun, Yann

AU - Huang, Fu Jie

PY - 2005

Y1 - 2005

N2 - Probabilistic graphical models associate a probability to each configuration of the relevant variables. Energy-based models (EBM) associate an energy to those configurations, eliminating the need for proper normalization of probability distributions. Making a decision (an inference) with an EBM consists in comparing the energies associated with various configurations of the variable to be predicted, and choosing the one with the smallest energy. Such systems must be trained discriminatively to associate low energies to the desired configurations and higher energies to un-desired configurations. A wide variety of loss function can be used for this purpose. We give sufficient conditions that a loss function should satisfy so that its minimization will cause the system to approach to desired behavior. We give many specific examples of suitable loss functions, and show an application to object recognition in images.

AB - Probabilistic graphical models associate a probability to each configuration of the relevant variables. Energy-based models (EBM) associate an energy to those configurations, eliminating the need for proper normalization of probability distributions. Making a decision (an inference) with an EBM consists in comparing the energies associated with various configurations of the variable to be predicted, and choosing the one with the smallest energy. Such systems must be trained discriminatively to associate low energies to the desired configurations and higher energies to un-desired configurations. A wide variety of loss function can be used for this purpose. We give sufficient conditions that a loss function should satisfy so that its minimization will cause the system to approach to desired behavior. We give many specific examples of suitable loss functions, and show an application to object recognition in images.

UR - http://www.scopus.com/inward/record.url?scp=84862617580&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84862617580&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84862617580

SN - 097273581X

SN - 9780972735810

SP - 206

EP - 213

BT - AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics

ER -