Distributed strategic learning with application to network security

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider in this paper a class of two-player nonzero-sum stochastic games with incomplete information. We develop fully distributed reinforcement learning algorithms, which require for each player a minimal amount of information regarding the other player. At each time, each player can be in an active mode or in a sleep mode. If a player is in an active mode, she updates her strategy and estimates of unknown quantities using a specific pure or hybrid learning pattern. We use stochastic approximation techniques to show that, under appropriate conditions, the pure or hybrid learning schemes with random updates can be studied using their deterministic ordinary differential equation (ODE) counterparts. Convergence to state-independent equilibria is analyzed under specific payoff functions. Results are applied to a class of security games in which the attacker and the defender adopt different learning schemes and update their strategies at random times.

Original languageEnglish (US)
Title of host publicationProceedings of the 2011 American Control Conference, ACC 2011
Pages4057-4062
Number of pages6
StatePublished - 2011
Event2011 American Control Conference, ACC 2011 - San Francisco, CA, United States
Duration: Jun 29 2011Jul 1 2011

Other

Other2011 American Control Conference, ACC 2011
CountryUnited States
CitySan Francisco, CA
Period6/29/117/1/11

Fingerprint

Network security
Reinforcement learning
Ordinary differential equations
Learning algorithms
Sleep

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Zhu, Q., Hamidou, T., & Başar, T. (2011). Distributed strategic learning with application to network security. In Proceedings of the 2011 American Control Conference, ACC 2011 (pp. 4057-4062). [5991373]

Distributed strategic learning with application to network security. / Zhu, Quanyan; Hamidou, Tembine; Başar, Tamer.

Proceedings of the 2011 American Control Conference, ACC 2011. 2011. p. 4057-4062 5991373.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhu, Q, Hamidou, T & Başar, T 2011, Distributed strategic learning with application to network security. in Proceedings of the 2011 American Control Conference, ACC 2011., 5991373, pp. 4057-4062, 2011 American Control Conference, ACC 2011, San Francisco, CA, United States, 6/29/11.
Zhu Q, Hamidou T, Başar T. Distributed strategic learning with application to network security. In Proceedings of the 2011 American Control Conference, ACC 2011. 2011. p. 4057-4062. 5991373
Zhu, Quanyan ; Hamidou, Tembine ; Başar, Tamer. / Distributed strategic learning with application to network security. Proceedings of the 2011 American Control Conference, ACC 2011. 2011. pp. 4057-4062
@inproceedings{4e886e0d95674108874ede8850f9d90b,
title = "Distributed strategic learning with application to network security",
abstract = "We consider in this paper a class of two-player nonzero-sum stochastic games with incomplete information. We develop fully distributed reinforcement learning algorithms, which require for each player a minimal amount of information regarding the other player. At each time, each player can be in an active mode or in a sleep mode. If a player is in an active mode, she updates her strategy and estimates of unknown quantities using a specific pure or hybrid learning pattern. We use stochastic approximation techniques to show that, under appropriate conditions, the pure or hybrid learning schemes with random updates can be studied using their deterministic ordinary differential equation (ODE) counterparts. Convergence to state-independent equilibria is analyzed under specific payoff functions. Results are applied to a class of security games in which the attacker and the defender adopt different learning schemes and update their strategies at random times.",
author = "Quanyan Zhu and Tembine Hamidou and Tamer Başar",
year = "2011",
language = "English (US)",
isbn = "9781457700804",
pages = "4057--4062",
booktitle = "Proceedings of the 2011 American Control Conference, ACC 2011",

}

TY - GEN

T1 - Distributed strategic learning with application to network security

AU - Zhu, Quanyan

AU - Hamidou, Tembine

AU - Başar, Tamer

PY - 2011

Y1 - 2011

N2 - We consider in this paper a class of two-player nonzero-sum stochastic games with incomplete information. We develop fully distributed reinforcement learning algorithms, which require for each player a minimal amount of information regarding the other player. At each time, each player can be in an active mode or in a sleep mode. If a player is in an active mode, she updates her strategy and estimates of unknown quantities using a specific pure or hybrid learning pattern. We use stochastic approximation techniques to show that, under appropriate conditions, the pure or hybrid learning schemes with random updates can be studied using their deterministic ordinary differential equation (ODE) counterparts. Convergence to state-independent equilibria is analyzed under specific payoff functions. Results are applied to a class of security games in which the attacker and the defender adopt different learning schemes and update their strategies at random times.

AB - We consider in this paper a class of two-player nonzero-sum stochastic games with incomplete information. We develop fully distributed reinforcement learning algorithms, which require for each player a minimal amount of information regarding the other player. At each time, each player can be in an active mode or in a sleep mode. If a player is in an active mode, she updates her strategy and estimates of unknown quantities using a specific pure or hybrid learning pattern. We use stochastic approximation techniques to show that, under appropriate conditions, the pure or hybrid learning schemes with random updates can be studied using their deterministic ordinary differential equation (ODE) counterparts. Convergence to state-independent equilibria is analyzed under specific payoff functions. Results are applied to a class of security games in which the attacker and the defender adopt different learning schemes and update their strategies at random times.

UR - http://www.scopus.com/inward/record.url?scp=80053140894&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053140894&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:80053140894

SN - 9781457700804

SP - 4057

EP - 4062

BT - Proceedings of the 2011 American Control Conference, ACC 2011

ER -