A hybrid neural network-latent topic model

Li Wan, Leo Zhu, Rob Fergus

Research output: Contribution to journalArticle

Abstract

This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a lowdimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.

Original languageEnglish (US)
Pages (from-to)1287-1294
Number of pages8
JournalJournal of Machine Learning Research
Volume22
StatePublished - 2012

Fingerprint

Neural Networks
Neural networks
Hybrid Model
Discriminative Training
Model
Extractor
Baseline
Entire
Model-based

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability

Cite this

A hybrid neural network-latent topic model. / Wan, Li; Zhu, Leo; Fergus, Rob.

In: Journal of Machine Learning Research, Vol. 22, 2012, p. 1287-1294.

Research output: Contribution to journalArticle

@article{76a989bf928349a6960c737f42318eb8,
title = "A hybrid neural network-latent topic model",
abstract = "This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a lowdimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.",
author = "Li Wan and Leo Zhu and Rob Fergus",
year = "2012",
language = "English (US)",
volume = "22",
pages = "1287--1294",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

TY - JOUR

T1 - A hybrid neural network-latent topic model

AU - Wan, Li

AU - Zhu, Leo

AU - Fergus, Rob

PY - 2012

Y1 - 2012

N2 - This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a lowdimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.

AB - This paper introduces a hybrid model that combines a neural network with a latent topic model. The neural network provides a lowdimensional embedding for the input data, whose subsequent distribution is captured by the topic model. The neural network thus acts as a trainable feature extractor while the topic model captures the group structure of the data. Following an initial pretraining phase to separately initialize each part of the model, a unified training scheme is introduced that allows for discriminative training of the entire model. The approach is evaluated on visual data in scene classification task, where the hybrid model is shown to outperform models based solely on neural networks or topic models, as well as other baseline methods.

UR - http://www.scopus.com/inward/record.url?scp=84954233874&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84954233874&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:84954233874

VL - 22

SP - 1287

EP - 1294

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -