Modeling others using oneself in multi-agent reinforcement learning

Roberta Raileanu, Emily Denton, Arthur Szlam, Robert Fergus

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider the multi-agent reinforcement learning setting with imperfect information. The reward function depends on the hidden goals of both agents, so the agents must infer the other players' goals from their observed behavior in order to maximize their returns. We propose a new approach for learning in these domains: Self Other-Modeling (SOM), in which an agent uses its own policy to predict the other agent's actions and update its belief of their hidden goal in an online manner. We evaluate this approach on three different tasks and show that the agents are able to learn better policies using their estimate of the other players' goals, in both cooperative and competitive settings.

Original languageEnglish (US)
Title of host publication35th International Conference on Machine Learning, ICML 2018
EditorsAndreas Krause, Jennifer Dy
PublisherInternational Machine Learning Society (IMLS)
Pages6779-6788
Number of pages10
Volume10
ISBN (Electronic)9781510867963
StatePublished - Jan 1 2018
Event35th International Conference on Machine Learning, ICML 2018 - Stockholm, Sweden
Duration: Jul 10 2018Jul 15 2018

Other

Other35th International Conference on Machine Learning, ICML 2018
CountrySweden
CityStockholm
Period7/10/187/15/18

Fingerprint

Reinforcement learning

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Human-Computer Interaction
  • Software

Cite this

Raileanu, R., Denton, E., Szlam, A., & Fergus, R. (2018). Modeling others using oneself in multi-agent reinforcement learning. In A. Krause, & J. Dy (Eds.), 35th International Conference on Machine Learning, ICML 2018 (Vol. 10, pp. 6779-6788). International Machine Learning Society (IMLS).

Modeling others using oneself in multi-agent reinforcement learning. / Raileanu, Roberta; Denton, Emily; Szlam, Arthur; Fergus, Robert.

35th International Conference on Machine Learning, ICML 2018. ed. / Andreas Krause; Jennifer Dy. Vol. 10 International Machine Learning Society (IMLS), 2018. p. 6779-6788.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Raileanu, R, Denton, E, Szlam, A & Fergus, R 2018, Modeling others using oneself in multi-agent reinforcement learning. in A Krause & J Dy (eds), 35th International Conference on Machine Learning, ICML 2018. vol. 10, International Machine Learning Society (IMLS), pp. 6779-6788, 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, 7/10/18.
Raileanu R, Denton E, Szlam A, Fergus R. Modeling others using oneself in multi-agent reinforcement learning. In Krause A, Dy J, editors, 35th International Conference on Machine Learning, ICML 2018. Vol. 10. International Machine Learning Society (IMLS). 2018. p. 6779-6788
Raileanu, Roberta ; Denton, Emily ; Szlam, Arthur ; Fergus, Robert. / Modeling others using oneself in multi-agent reinforcement learning. 35th International Conference on Machine Learning, ICML 2018. editor / Andreas Krause ; Jennifer Dy. Vol. 10 International Machine Learning Society (IMLS), 2018. pp. 6779-6788
@inproceedings{2174b58647874d1b9985208723688f0e,
title = "Modeling others using oneself in multi-agent reinforcement learning",
abstract = "We consider the multi-agent reinforcement learning setting with imperfect information. The reward function depends on the hidden goals of both agents, so the agents must infer the other players' goals from their observed behavior in order to maximize their returns. We propose a new approach for learning in these domains: Self Other-Modeling (SOM), in which an agent uses its own policy to predict the other agent's actions and update its belief of their hidden goal in an online manner. We evaluate this approach on three different tasks and show that the agents are able to learn better policies using their estimate of the other players' goals, in both cooperative and competitive settings.",
author = "Roberta Raileanu and Emily Denton and Arthur Szlam and Robert Fergus",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
volume = "10",
pages = "6779--6788",
editor = "Andreas Krause and Jennifer Dy",
booktitle = "35th International Conference on Machine Learning, ICML 2018",
publisher = "International Machine Learning Society (IMLS)",

}

TY - GEN

T1 - Modeling others using oneself in multi-agent reinforcement learning

AU - Raileanu, Roberta

AU - Denton, Emily

AU - Szlam, Arthur

AU - Fergus, Robert

PY - 2018/1/1

Y1 - 2018/1/1

N2 - We consider the multi-agent reinforcement learning setting with imperfect information. The reward function depends on the hidden goals of both agents, so the agents must infer the other players' goals from their observed behavior in order to maximize their returns. We propose a new approach for learning in these domains: Self Other-Modeling (SOM), in which an agent uses its own policy to predict the other agent's actions and update its belief of their hidden goal in an online manner. We evaluate this approach on three different tasks and show that the agents are able to learn better policies using their estimate of the other players' goals, in both cooperative and competitive settings.

AB - We consider the multi-agent reinforcement learning setting with imperfect information. The reward function depends on the hidden goals of both agents, so the agents must infer the other players' goals from their observed behavior in order to maximize their returns. We propose a new approach for learning in these domains: Self Other-Modeling (SOM), in which an agent uses its own policy to predict the other agent's actions and update its belief of their hidden goal in an online manner. We evaluate this approach on three different tasks and show that the agents are able to learn better policies using their estimate of the other players' goals, in both cooperative and competitive settings.

UR - http://www.scopus.com/inward/record.url?scp=85057346427&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057346427&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85057346427

VL - 10

SP - 6779

EP - 6788

BT - 35th International Conference on Machine Learning, ICML 2018

A2 - Krause, Andreas

A2 - Dy, Jennifer

PB - International Machine Learning Society (IMLS)

ER -