Domain adaptation: Learning bounds and algorithms

Yishay Mansour, Mehryar Mohri, Afshin Rostamizadeh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper addresses the general problem of domain adaptation which arises in a variety of applications where the distribution of the labeled sample available somewhat differs from that of the test data. Building on previous work by Ben-David et al. (2007), we introduce a novel distance between distributions, discrepancy distance, that is tailored to adaptation problems with arbitrary loss functions. We give Rademacher complexity bounds for estimating the discrepancy distance from finite samples for different loss functions. Using this distance, we derive new generalization bounds for domain adaptation for a wide family of loss functions. We also present a series of novel adaptation bounds for large classes of regularization-based algorithms, including support vector machines and kernel ridge regression based on the empirical discrepancy. This motivates our analysis of the problem of minimizing the empirical discrepancy for various loss functions for which we also give several algorithms. We report the results of preliminary experiments that demonstrate the benefits of our discrepancy minimization algorithms for domain adaptation.

Original languageEnglish (US)
Title of host publicationCOLT 2009 - The 22nd Conference on Learning Theory
StatePublished - 2009
Event22nd Conference on Learning Theory, COLT 2009 - Montreal, QC, Canada
Duration: Jun 18 2009Jun 21 2009

Other

Other22nd Conference on Learning Theory, COLT 2009
CountryCanada
CityMontreal, QC
Period6/18/096/21/09

Fingerprint

learning
loss of function
regression
experiment

ASJC Scopus subject areas

  • Education

Cite this

Mansour, Y., Mohri, M., & Rostamizadeh, A. (2009). Domain adaptation: Learning bounds and algorithms. In COLT 2009 - The 22nd Conference on Learning Theory

Domain adaptation : Learning bounds and algorithms. / Mansour, Yishay; Mohri, Mehryar; Rostamizadeh, Afshin.

COLT 2009 - The 22nd Conference on Learning Theory. 2009.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mansour, Y, Mohri, M & Rostamizadeh, A 2009, Domain adaptation: Learning bounds and algorithms. in COLT 2009 - The 22nd Conference on Learning Theory. 22nd Conference on Learning Theory, COLT 2009, Montreal, QC, Canada, 6/18/09.
Mansour Y, Mohri M, Rostamizadeh A. Domain adaptation: Learning bounds and algorithms. In COLT 2009 - The 22nd Conference on Learning Theory. 2009
Mansour, Yishay ; Mohri, Mehryar ; Rostamizadeh, Afshin. / Domain adaptation : Learning bounds and algorithms. COLT 2009 - The 22nd Conference on Learning Theory. 2009.
@inproceedings{3f27e85748124342b07b44b38cea884e,
title = "Domain adaptation: Learning bounds and algorithms",
abstract = "This paper addresses the general problem of domain adaptation which arises in a variety of applications where the distribution of the labeled sample available somewhat differs from that of the test data. Building on previous work by Ben-David et al. (2007), we introduce a novel distance between distributions, discrepancy distance, that is tailored to adaptation problems with arbitrary loss functions. We give Rademacher complexity bounds for estimating the discrepancy distance from finite samples for different loss functions. Using this distance, we derive new generalization bounds for domain adaptation for a wide family of loss functions. We also present a series of novel adaptation bounds for large classes of regularization-based algorithms, including support vector machines and kernel ridge regression based on the empirical discrepancy. This motivates our analysis of the problem of minimizing the empirical discrepancy for various loss functions for which we also give several algorithms. We report the results of preliminary experiments that demonstrate the benefits of our discrepancy minimization algorithms for domain adaptation.",
author = "Yishay Mansour and Mehryar Mohri and Afshin Rostamizadeh",
year = "2009",
language = "English (US)",
booktitle = "COLT 2009 - The 22nd Conference on Learning Theory",

}

TY - GEN

T1 - Domain adaptation

T2 - Learning bounds and algorithms

AU - Mansour, Yishay

AU - Mohri, Mehryar

AU - Rostamizadeh, Afshin

PY - 2009

Y1 - 2009

N2 - This paper addresses the general problem of domain adaptation which arises in a variety of applications where the distribution of the labeled sample available somewhat differs from that of the test data. Building on previous work by Ben-David et al. (2007), we introduce a novel distance between distributions, discrepancy distance, that is tailored to adaptation problems with arbitrary loss functions. We give Rademacher complexity bounds for estimating the discrepancy distance from finite samples for different loss functions. Using this distance, we derive new generalization bounds for domain adaptation for a wide family of loss functions. We also present a series of novel adaptation bounds for large classes of regularization-based algorithms, including support vector machines and kernel ridge regression based on the empirical discrepancy. This motivates our analysis of the problem of minimizing the empirical discrepancy for various loss functions for which we also give several algorithms. We report the results of preliminary experiments that demonstrate the benefits of our discrepancy minimization algorithms for domain adaptation.

AB - This paper addresses the general problem of domain adaptation which arises in a variety of applications where the distribution of the labeled sample available somewhat differs from that of the test data. Building on previous work by Ben-David et al. (2007), we introduce a novel distance between distributions, discrepancy distance, that is tailored to adaptation problems with arbitrary loss functions. We give Rademacher complexity bounds for estimating the discrepancy distance from finite samples for different loss functions. Using this distance, we derive new generalization bounds for domain adaptation for a wide family of loss functions. We also present a series of novel adaptation bounds for large classes of regularization-based algorithms, including support vector machines and kernel ridge regression based on the empirical discrepancy. This motivates our analysis of the problem of minimizing the empirical discrepancy for various loss functions for which we also give several algorithms. We report the results of preliminary experiments that demonstrate the benefits of our discrepancy minimization algorithms for domain adaptation.

UR - http://www.scopus.com/inward/record.url?scp=84898072330&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898072330&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84898072330

BT - COLT 2009 - The 22nd Conference on Learning Theory

ER -