Markov decision evolutionary games with expected average fitness

E. Altman, Y. Hayel, Tembine Hamidou, R. El-Azouzi

Research output: Contribution to journalArticle

Abstract

Aim: To model and characterize evolutionary games where individuals have states that are described by controlled Markov chains. The action of an individual in a local interaction with another randomly selected individual determines not only the instantaneous fitness but also its probability to move to another state. The goal of a player is to maximize its time average fitness. Mathematical methods: The main mathematical tool is occupation measures (expected frequencies of states and actions). This tool is a central one in the theory of Markov decision processes. We make use of the geometric properties of the set of achievable occupation measures. Key assumption: Under any pure stationary policy of an individual, its Markov chain has a single ergodic class of states. Results: We define and characterize a new concept of evolutionarily stable strategies based on the concept of 'occupation measures'. We relate this set to the concept of evolutionarily stable set (ESSet). We present a way to transform the new type of evolutionary games into standard ones. Applying this novel framework to energy control in wireless networks, we show the existence of an occupation measure ESS (OMESS).

Original languageEnglish (US)
Pages (from-to)677-689
Number of pages13
JournalEvolutionary Ecology Research
Volume11
Issue number4
StatePublished - May 1 2009

Fingerprint

occupation
fitness
evolutionarily stable strategy
Markov chain
numerical method
transform
decision
energy
methodology

Keywords

  • Energy control in wireless networks
  • Evolutionarily stable strategy
  • Evolutionary games
  • Markov decision process
  • Occupation measure

ASJC Scopus subject areas

  • Ecology, Evolution, Behavior and Systematics

Cite this

Markov decision evolutionary games with expected average fitness. / Altman, E.; Hayel, Y.; Hamidou, Tembine; El-Azouzi, R.

In: Evolutionary Ecology Research, Vol. 11, No. 4, 01.05.2009, p. 677-689.

Research output: Contribution to journalArticle

Altman, E, Hayel, Y, Hamidou, T & El-Azouzi, R 2009, 'Markov decision evolutionary games with expected average fitness', Evolutionary Ecology Research, vol. 11, no. 4, pp. 677-689.
Altman, E. ; Hayel, Y. ; Hamidou, Tembine ; El-Azouzi, R. / Markov decision evolutionary games with expected average fitness. In: Evolutionary Ecology Research. 2009 ; Vol. 11, No. 4. pp. 677-689.
@article{514b637789d34e7099ab156e3fc2ad36,
title = "Markov decision evolutionary games with expected average fitness",
abstract = "Aim: To model and characterize evolutionary games where individuals have states that are described by controlled Markov chains. The action of an individual in a local interaction with another randomly selected individual determines not only the instantaneous fitness but also its probability to move to another state. The goal of a player is to maximize its time average fitness. Mathematical methods: The main mathematical tool is occupation measures (expected frequencies of states and actions). This tool is a central one in the theory of Markov decision processes. We make use of the geometric properties of the set of achievable occupation measures. Key assumption: Under any pure stationary policy of an individual, its Markov chain has a single ergodic class of states. Results: We define and characterize a new concept of evolutionarily stable strategies based on the concept of 'occupation measures'. We relate this set to the concept of evolutionarily stable set (ESSet). We present a way to transform the new type of evolutionary games into standard ones. Applying this novel framework to energy control in wireless networks, we show the existence of an occupation measure ESS (OMESS).",
keywords = "Energy control in wireless networks, Evolutionarily stable strategy, Evolutionary games, Markov decision process, Occupation measure",
author = "E. Altman and Y. Hayel and Tembine Hamidou and R. El-Azouzi",
year = "2009",
month = "5",
day = "1",
language = "English (US)",
volume = "11",
pages = "677--689",
journal = "Evolutionary Ecology Research",
issn = "1522-0613",
publisher = "Evolutionary Ecology Research",
number = "4",

}

TY - JOUR

T1 - Markov decision evolutionary games with expected average fitness

AU - Altman, E.

AU - Hayel, Y.

AU - Hamidou, Tembine

AU - El-Azouzi, R.

PY - 2009/5/1

Y1 - 2009/5/1

N2 - Aim: To model and characterize evolutionary games where individuals have states that are described by controlled Markov chains. The action of an individual in a local interaction with another randomly selected individual determines not only the instantaneous fitness but also its probability to move to another state. The goal of a player is to maximize its time average fitness. Mathematical methods: The main mathematical tool is occupation measures (expected frequencies of states and actions). This tool is a central one in the theory of Markov decision processes. We make use of the geometric properties of the set of achievable occupation measures. Key assumption: Under any pure stationary policy of an individual, its Markov chain has a single ergodic class of states. Results: We define and characterize a new concept of evolutionarily stable strategies based on the concept of 'occupation measures'. We relate this set to the concept of evolutionarily stable set (ESSet). We present a way to transform the new type of evolutionary games into standard ones. Applying this novel framework to energy control in wireless networks, we show the existence of an occupation measure ESS (OMESS).

AB - Aim: To model and characterize evolutionary games where individuals have states that are described by controlled Markov chains. The action of an individual in a local interaction with another randomly selected individual determines not only the instantaneous fitness but also its probability to move to another state. The goal of a player is to maximize its time average fitness. Mathematical methods: The main mathematical tool is occupation measures (expected frequencies of states and actions). This tool is a central one in the theory of Markov decision processes. We make use of the geometric properties of the set of achievable occupation measures. Key assumption: Under any pure stationary policy of an individual, its Markov chain has a single ergodic class of states. Results: We define and characterize a new concept of evolutionarily stable strategies based on the concept of 'occupation measures'. We relate this set to the concept of evolutionarily stable set (ESSet). We present a way to transform the new type of evolutionary games into standard ones. Applying this novel framework to energy control in wireless networks, we show the existence of an occupation measure ESS (OMESS).

KW - Energy control in wireless networks

KW - Evolutionarily stable strategy

KW - Evolutionary games

KW - Markov decision process

KW - Occupation measure

UR - http://www.scopus.com/inward/record.url?scp=70350154803&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70350154803&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:70350154803

VL - 11

SP - 677

EP - 689

JO - Evolutionary Ecology Research

JF - Evolutionary Ecology Research

SN - 1522-0613

IS - 4

ER -