Training convolutional networks with noisy labels

Sainbayar Sukhbaatar, Joan Bruna Estrach, Manohar Paluri, Lubomir Bourdev, Robert Fergus

Research output: Contribution to conferencePaper

Abstract

The availability of large labeled datasets has allowed Convolutional Network models to achieve impressive recognition results. However, in many settings manual annotation of the data is impractical; instead our data has noisy labels, i.e. there is some freely available label for each image which may or may not be accurate. In this paper, we explore the performance of discriminatively-trained Convnets when trained on such noisy data. We introduce an extra noise layer into the network which adapts the network outputs to match the noisy label distribution. The parameters of this noise layer can be estimated as part of the training process and involve simple modifications to current training infrastructures for deep networks. We demonstrate the approaches on several datasets, including large scale experiments on the ImageNet classification benchmark.

Original languageEnglish (US)
StatePublished - Jan 1 2015
Event3rd International Conference on Learning Representations, ICLR 2015 - San Diego, United States
Duration: May 7 2015May 9 2015

Conference

Conference3rd International Conference on Learning Representations, ICLR 2015
CountryUnited States
CitySan Diego
Period5/7/155/9/15

Fingerprint

Labels
Availability
infrastructure
experiment
Experiments
performance
Layer
Network Model
Benchmark
Annotation
Experiment

ASJC Scopus subject areas

  • Education
  • Linguistics and Language
  • Language and Linguistics
  • Computer Science Applications

Cite this

Sukhbaatar, S., Bruna Estrach, J., Paluri, M., Bourdev, L., & Fergus, R. (2015). Training convolutional networks with noisy labels. Paper presented at 3rd International Conference on Learning Representations, ICLR 2015, San Diego, United States.

Training convolutional networks with noisy labels. / Sukhbaatar, Sainbayar; Bruna Estrach, Joan; Paluri, Manohar; Bourdev, Lubomir; Fergus, Robert.

2015. Paper presented at 3rd International Conference on Learning Representations, ICLR 2015, San Diego, United States.

Research output: Contribution to conferencePaper

Sukhbaatar, S, Bruna Estrach, J, Paluri, M, Bourdev, L & Fergus, R 2015, 'Training convolutional networks with noisy labels' Paper presented at 3rd International Conference on Learning Representations, ICLR 2015, San Diego, United States, 5/7/15 - 5/9/15, .
Sukhbaatar S, Bruna Estrach J, Paluri M, Bourdev L, Fergus R. Training convolutional networks with noisy labels. 2015. Paper presented at 3rd International Conference on Learning Representations, ICLR 2015, San Diego, United States.
Sukhbaatar, Sainbayar ; Bruna Estrach, Joan ; Paluri, Manohar ; Bourdev, Lubomir ; Fergus, Robert. / Training convolutional networks with noisy labels. Paper presented at 3rd International Conference on Learning Representations, ICLR 2015, San Diego, United States.
@conference{c51d68a3106242f08ed001d0c46320b3,
title = "Training convolutional networks with noisy labels",
abstract = "The availability of large labeled datasets has allowed Convolutional Network models to achieve impressive recognition results. However, in many settings manual annotation of the data is impractical; instead our data has noisy labels, i.e. there is some freely available label for each image which may or may not be accurate. In this paper, we explore the performance of discriminatively-trained Convnets when trained on such noisy data. We introduce an extra noise layer into the network which adapts the network outputs to match the noisy label distribution. The parameters of this noise layer can be estimated as part of the training process and involve simple modifications to current training infrastructures for deep networks. We demonstrate the approaches on several datasets, including large scale experiments on the ImageNet classification benchmark.",
author = "Sainbayar Sukhbaatar and {Bruna Estrach}, Joan and Manohar Paluri and Lubomir Bourdev and Robert Fergus",
year = "2015",
month = "1",
day = "1",
language = "English (US)",
note = "3rd International Conference on Learning Representations, ICLR 2015 ; Conference date: 07-05-2015 Through 09-05-2015",

}

TY - CONF

T1 - Training convolutional networks with noisy labels

AU - Sukhbaatar, Sainbayar

AU - Bruna Estrach, Joan

AU - Paluri, Manohar

AU - Bourdev, Lubomir

AU - Fergus, Robert

PY - 2015/1/1

Y1 - 2015/1/1

N2 - The availability of large labeled datasets has allowed Convolutional Network models to achieve impressive recognition results. However, in many settings manual annotation of the data is impractical; instead our data has noisy labels, i.e. there is some freely available label for each image which may or may not be accurate. In this paper, we explore the performance of discriminatively-trained Convnets when trained on such noisy data. We introduce an extra noise layer into the network which adapts the network outputs to match the noisy label distribution. The parameters of this noise layer can be estimated as part of the training process and involve simple modifications to current training infrastructures for deep networks. We demonstrate the approaches on several datasets, including large scale experiments on the ImageNet classification benchmark.

AB - The availability of large labeled datasets has allowed Convolutional Network models to achieve impressive recognition results. However, in many settings manual annotation of the data is impractical; instead our data has noisy labels, i.e. there is some freely available label for each image which may or may not be accurate. In this paper, we explore the performance of discriminatively-trained Convnets when trained on such noisy data. We introduce an extra noise layer into the network which adapts the network outputs to match the noisy label distribution. The parameters of this noise layer can be estimated as part of the training process and involve simple modifications to current training infrastructures for deep networks. We demonstrate the approaches on several datasets, including large scale experiments on the ImageNet classification benchmark.

UR - http://www.scopus.com/inward/record.url?scp=84966607207&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84966607207&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:84966607207

ER -