Mean field stochastic games

Convergence, Q/H-learning and optimality

Tembine Hamidou

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    We consider a class of stochastic games with finite number of resource states, individual states and actions per states. At each stage, a random set of players interact. The states and the actions of all the interacting players determine together the instantaneous payoffs and the transitions to the next states. We study the convergence of the stochastic game with variable set of interacting players when the total number of possible players grow without bound. We provide sufficient conditions for mean field convergence. We characterize the mean field payoff optimality by solutions of a coupled system of backward-forward equations. The limiting games are equivalent to discrete time anonymous sequential population games or to differential population games. Using multidimensional diffusion processes, a general mean field convergence to coupled stochastic differential equation is given. Finally, the computation of mean field equilibria is addressed using Q/H learning.

    Original languageEnglish (US)
    Title of host publicationProceedings of the 2011 American Control Conference, ACC 2011
    Pages2423-2428
    Number of pages6
    StatePublished - Sep 29 2011
    Event2011 American Control Conference, ACC 2011 - San Francisco, CA, United States
    Duration: Jun 29 2011Jul 1 2011

    Other

    Other2011 American Control Conference, ACC 2011
    CountryUnited States
    CitySan Francisco, CA
    Period6/29/117/1/11

    Fingerprint

    Differential equations

    ASJC Scopus subject areas

    • Electrical and Electronic Engineering

    Cite this

    Hamidou, T. (2011). Mean field stochastic games: Convergence, Q/H-learning and optimality. In Proceedings of the 2011 American Control Conference, ACC 2011 (pp. 2423-2428). [5991087]

    Mean field stochastic games : Convergence, Q/H-learning and optimality. / Hamidou, Tembine.

    Proceedings of the 2011 American Control Conference, ACC 2011. 2011. p. 2423-2428 5991087.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Hamidou, T 2011, Mean field stochastic games: Convergence, Q/H-learning and optimality. in Proceedings of the 2011 American Control Conference, ACC 2011., 5991087, pp. 2423-2428, 2011 American Control Conference, ACC 2011, San Francisco, CA, United States, 6/29/11.
    Hamidou T. Mean field stochastic games: Convergence, Q/H-learning and optimality. In Proceedings of the 2011 American Control Conference, ACC 2011. 2011. p. 2423-2428. 5991087
    Hamidou, Tembine. / Mean field stochastic games : Convergence, Q/H-learning and optimality. Proceedings of the 2011 American Control Conference, ACC 2011. 2011. pp. 2423-2428
    @inproceedings{a5f5abd18ebc4a93a5eb2edee2da38d0,
    title = "Mean field stochastic games: Convergence, Q/H-learning and optimality",
    abstract = "We consider a class of stochastic games with finite number of resource states, individual states and actions per states. At each stage, a random set of players interact. The states and the actions of all the interacting players determine together the instantaneous payoffs and the transitions to the next states. We study the convergence of the stochastic game with variable set of interacting players when the total number of possible players grow without bound. We provide sufficient conditions for mean field convergence. We characterize the mean field payoff optimality by solutions of a coupled system of backward-forward equations. The limiting games are equivalent to discrete time anonymous sequential population games or to differential population games. Using multidimensional diffusion processes, a general mean field convergence to coupled stochastic differential equation is given. Finally, the computation of mean field equilibria is addressed using Q/H learning.",
    author = "Tembine Hamidou",
    year = "2011",
    month = "9",
    day = "29",
    language = "English (US)",
    isbn = "9781457700804",
    pages = "2423--2428",
    booktitle = "Proceedings of the 2011 American Control Conference, ACC 2011",

    }

    TY - GEN

    T1 - Mean field stochastic games

    T2 - Convergence, Q/H-learning and optimality

    AU - Hamidou, Tembine

    PY - 2011/9/29

    Y1 - 2011/9/29

    N2 - We consider a class of stochastic games with finite number of resource states, individual states and actions per states. At each stage, a random set of players interact. The states and the actions of all the interacting players determine together the instantaneous payoffs and the transitions to the next states. We study the convergence of the stochastic game with variable set of interacting players when the total number of possible players grow without bound. We provide sufficient conditions for mean field convergence. We characterize the mean field payoff optimality by solutions of a coupled system of backward-forward equations. The limiting games are equivalent to discrete time anonymous sequential population games or to differential population games. Using multidimensional diffusion processes, a general mean field convergence to coupled stochastic differential equation is given. Finally, the computation of mean field equilibria is addressed using Q/H learning.

    AB - We consider a class of stochastic games with finite number of resource states, individual states and actions per states. At each stage, a random set of players interact. The states and the actions of all the interacting players determine together the instantaneous payoffs and the transitions to the next states. We study the convergence of the stochastic game with variable set of interacting players when the total number of possible players grow without bound. We provide sufficient conditions for mean field convergence. We characterize the mean field payoff optimality by solutions of a coupled system of backward-forward equations. The limiting games are equivalent to discrete time anonymous sequential population games or to differential population games. Using multidimensional diffusion processes, a general mean field convergence to coupled stochastic differential equation is given. Finally, the computation of mean field equilibria is addressed using Q/H learning.

    UR - http://www.scopus.com/inward/record.url?scp=80053146604&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=80053146604&partnerID=8YFLogxK

    M3 - Conference contribution

    SN - 9781457700804

    SP - 2423

    EP - 2428

    BT - Proceedings of the 2011 American Control Conference, ACC 2011

    ER -