Distributionally Robust Games

Wasserstein Metric

Jian Gao, Tembine Hamidou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep generative models are powerful but difficult to train due to its instability, saturation problem and high dimensional data distribution. This paper introduces a game theory framework with Wasserstein metric to train generative models, in which the unknown data distribution is learned by dynamically optimizing the worst-case payoff. In the game, two types of players work on opposite objectives to solve a minimax problem. The defenders explore the Wasserstein neighborhood of real data to generate a set of hard samples which have the maximum distance from the model distribution. The attackers update the model to fit for the hard set so as to minimize the discrepancy between model and data distributions. Instead of Kullback-Leibler divergence, we use Wasserstein distance to measure the similarity between distributions. The Wasserstein metric is a true distance with better topology in the parameter space, which improves the stability of training. We provide practical algorithms to train deep generative models, in which an encoder network is designed to learn the feature vector of the high dimensional data. The algorithm is tested on CelebA human face dataset and compared with the state-of-the-art generative models. Performance evaluation shows the training process is stable and converges fast. Our model can produce visual pleasing images which are closer to the real distribution in terms of Wasserstein distance.

Original languageEnglish (US)
Title of host publication2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Volume2018-July
ISBN (Electronic)9781509060146
DOIs
StatePublished - Oct 10 2018
Event2018 International Joint Conference on Neural Networks, IJCNN 2018 - Rio de Janeiro, Brazil
Duration: Jul 8 2018Jul 13 2018

Other

Other2018 International Joint Conference on Neural Networks, IJCNN 2018
CountryBrazil
CityRio de Janeiro
Period7/8/187/13/18

Fingerprint

Game theory
Topology

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this

Gao, J., & Hamidou, T. (2018). Distributionally Robust Games: Wasserstein Metric. In 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings (Vol. 2018-July). [8489636] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN.2018.8489636

Distributionally Robust Games : Wasserstein Metric. / Gao, Jian; Hamidou, Tembine.

2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. Vol. 2018-July Institute of Electrical and Electronics Engineers Inc., 2018. 8489636.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Gao, J & Hamidou, T 2018, Distributionally Robust Games: Wasserstein Metric. in 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. vol. 2018-July, 8489636, Institute of Electrical and Electronics Engineers Inc., 2018 International Joint Conference on Neural Networks, IJCNN 2018, Rio de Janeiro, Brazil, 7/8/18. https://doi.org/10.1109/IJCNN.2018.8489636
Gao J, Hamidou T. Distributionally Robust Games: Wasserstein Metric. In 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. Vol. 2018-July. Institute of Electrical and Electronics Engineers Inc. 2018. 8489636 https://doi.org/10.1109/IJCNN.2018.8489636
Gao, Jian ; Hamidou, Tembine. / Distributionally Robust Games : Wasserstein Metric. 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings. Vol. 2018-July Institute of Electrical and Electronics Engineers Inc., 2018.
@inproceedings{f545aa658adf42949189434945985f35,
title = "Distributionally Robust Games: Wasserstein Metric",
abstract = "Deep generative models are powerful but difficult to train due to its instability, saturation problem and high dimensional data distribution. This paper introduces a game theory framework with Wasserstein metric to train generative models, in which the unknown data distribution is learned by dynamically optimizing the worst-case payoff. In the game, two types of players work on opposite objectives to solve a minimax problem. The defenders explore the Wasserstein neighborhood of real data to generate a set of hard samples which have the maximum distance from the model distribution. The attackers update the model to fit for the hard set so as to minimize the discrepancy between model and data distributions. Instead of Kullback-Leibler divergence, we use Wasserstein distance to measure the similarity between distributions. The Wasserstein metric is a true distance with better topology in the parameter space, which improves the stability of training. We provide practical algorithms to train deep generative models, in which an encoder network is designed to learn the feature vector of the high dimensional data. The algorithm is tested on CelebA human face dataset and compared with the state-of-the-art generative models. Performance evaluation shows the training process is stable and converges fast. Our model can produce visual pleasing images which are closer to the real distribution in terms of Wasserstein distance.",
author = "Jian Gao and Tembine Hamidou",
year = "2018",
month = "10",
day = "10",
doi = "10.1109/IJCNN.2018.8489636",
language = "English (US)",
volume = "2018-July",
booktitle = "2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Distributionally Robust Games

T2 - Wasserstein Metric

AU - Gao, Jian

AU - Hamidou, Tembine

PY - 2018/10/10

Y1 - 2018/10/10

N2 - Deep generative models are powerful but difficult to train due to its instability, saturation problem and high dimensional data distribution. This paper introduces a game theory framework with Wasserstein metric to train generative models, in which the unknown data distribution is learned by dynamically optimizing the worst-case payoff. In the game, two types of players work on opposite objectives to solve a minimax problem. The defenders explore the Wasserstein neighborhood of real data to generate a set of hard samples which have the maximum distance from the model distribution. The attackers update the model to fit for the hard set so as to minimize the discrepancy between model and data distributions. Instead of Kullback-Leibler divergence, we use Wasserstein distance to measure the similarity between distributions. The Wasserstein metric is a true distance with better topology in the parameter space, which improves the stability of training. We provide practical algorithms to train deep generative models, in which an encoder network is designed to learn the feature vector of the high dimensional data. The algorithm is tested on CelebA human face dataset and compared with the state-of-the-art generative models. Performance evaluation shows the training process is stable and converges fast. Our model can produce visual pleasing images which are closer to the real distribution in terms of Wasserstein distance.

AB - Deep generative models are powerful but difficult to train due to its instability, saturation problem and high dimensional data distribution. This paper introduces a game theory framework with Wasserstein metric to train generative models, in which the unknown data distribution is learned by dynamically optimizing the worst-case payoff. In the game, two types of players work on opposite objectives to solve a minimax problem. The defenders explore the Wasserstein neighborhood of real data to generate a set of hard samples which have the maximum distance from the model distribution. The attackers update the model to fit for the hard set so as to minimize the discrepancy between model and data distributions. Instead of Kullback-Leibler divergence, we use Wasserstein distance to measure the similarity between distributions. The Wasserstein metric is a true distance with better topology in the parameter space, which improves the stability of training. We provide practical algorithms to train deep generative models, in which an encoder network is designed to learn the feature vector of the high dimensional data. The algorithm is tested on CelebA human face dataset and compared with the state-of-the-art generative models. Performance evaluation shows the training process is stable and converges fast. Our model can produce visual pleasing images which are closer to the real distribution in terms of Wasserstein distance.

UR - http://www.scopus.com/inward/record.url?scp=85056515214&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056515214&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2018.8489636

DO - 10.1109/IJCNN.2018.8489636

M3 - Conference contribution

VL - 2018-July

BT - 2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -