Universum prescription: Regularization using unlabeled data

Xiang Zhang, Yann LeCun

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper shows that simply prescribing "none of the above" labels to unlabeled data has a beneficial regularization effect to supervised learning. We call it universum prescription by the fact that the prescribed labels cannot be one of the supervised labels. In spite of its simplicity, universum prescription obtained competitive results in training deep convolutional networks for CIFAR-10, CIFAR-100, STL-10 and ImageNet datasets. A qualitative justification of these approaches using Rademacher complexity is presented. The effect of a regularization parameter - probability of sampling from unlabeled data - is also studied empirically.

Original languageEnglish (US)
Title of host publication31st AAAI Conference on Artificial Intelligence, AAAI 2017
PublisherAAAI press
Pages2907-2913
Number of pages7
StatePublished - 2017
Event31st AAAI Conference on Artificial Intelligence, AAAI 2017 - San Francisco, United States
Duration: Feb 4 2017Feb 10 2017

Other

Other31st AAAI Conference on Artificial Intelligence, AAAI 2017
CountryUnited States
CitySan Francisco
Period2/4/172/10/17

Fingerprint

Labels
Supervised learning
Sampling

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Zhang, X., & LeCun, Y. (2017). Universum prescription: Regularization using unlabeled data. In 31st AAAI Conference on Artificial Intelligence, AAAI 2017 (pp. 2907-2913). AAAI press.

Universum prescription : Regularization using unlabeled data. / Zhang, Xiang; LeCun, Yann.

31st AAAI Conference on Artificial Intelligence, AAAI 2017. AAAI press, 2017. p. 2907-2913.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhang, X & LeCun, Y 2017, Universum prescription: Regularization using unlabeled data. in 31st AAAI Conference on Artificial Intelligence, AAAI 2017. AAAI press, pp. 2907-2913, 31st AAAI Conference on Artificial Intelligence, AAAI 2017, San Francisco, United States, 2/4/17.
Zhang X, LeCun Y. Universum prescription: Regularization using unlabeled data. In 31st AAAI Conference on Artificial Intelligence, AAAI 2017. AAAI press. 2017. p. 2907-2913
Zhang, Xiang ; LeCun, Yann. / Universum prescription : Regularization using unlabeled data. 31st AAAI Conference on Artificial Intelligence, AAAI 2017. AAAI press, 2017. pp. 2907-2913
@inproceedings{e3c019988ee741298c6d4432ad4f32e7,
title = "Universum prescription: Regularization using unlabeled data",
abstract = "This paper shows that simply prescribing {"}none of the above{"} labels to unlabeled data has a beneficial regularization effect to supervised learning. We call it universum prescription by the fact that the prescribed labels cannot be one of the supervised labels. In spite of its simplicity, universum prescription obtained competitive results in training deep convolutional networks for CIFAR-10, CIFAR-100, STL-10 and ImageNet datasets. A qualitative justification of these approaches using Rademacher complexity is presented. The effect of a regularization parameter - probability of sampling from unlabeled data - is also studied empirically.",
author = "Xiang Zhang and Yann LeCun",
year = "2017",
language = "English (US)",
pages = "2907--2913",
booktitle = "31st AAAI Conference on Artificial Intelligence, AAAI 2017",
publisher = "AAAI press",

}

TY - GEN

T1 - Universum prescription

T2 - Regularization using unlabeled data

AU - Zhang, Xiang

AU - LeCun, Yann

PY - 2017

Y1 - 2017

N2 - This paper shows that simply prescribing "none of the above" labels to unlabeled data has a beneficial regularization effect to supervised learning. We call it universum prescription by the fact that the prescribed labels cannot be one of the supervised labels. In spite of its simplicity, universum prescription obtained competitive results in training deep convolutional networks for CIFAR-10, CIFAR-100, STL-10 and ImageNet datasets. A qualitative justification of these approaches using Rademacher complexity is presented. The effect of a regularization parameter - probability of sampling from unlabeled data - is also studied empirically.

AB - This paper shows that simply prescribing "none of the above" labels to unlabeled data has a beneficial regularization effect to supervised learning. We call it universum prescription by the fact that the prescribed labels cannot be one of the supervised labels. In spite of its simplicity, universum prescription obtained competitive results in training deep convolutional networks for CIFAR-10, CIFAR-100, STL-10 and ImageNet datasets. A qualitative justification of these approaches using Rademacher complexity is presented. The effect of a regularization parameter - probability of sampling from unlabeled data - is also studied empirically.

UR - http://www.scopus.com/inward/record.url?scp=85030478667&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85030478667&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85030478667

SP - 2907

EP - 2913

BT - 31st AAAI Conference on Artificial Intelligence, AAAI 2017

PB - AAAI press

ER -