Playing Multi-Action Adversarial Games: Online Evolutionary Planning versus Tree Search

Niels Justesen, Tobias Mahlmann, Sebastian Risi, Julian Togelius

    Research output: Contribution to journalArticle

    Abstract

    We address the problem of playing turn-based multi-action adversarial games, which include many strategy games with extremely high branching factors as players take multiple actions each turn. This leads to the breakdown of standard tree search methods, including Monte Carlo Tree Search (MCTS), as they become unable to reach a sufficient depth in the game tree. In this paper we introduce Online Evolutionary Planning (OEP) to address this challenge, which searches for combinations of actions to perform during a single turn guided by a fitness function that evaluates the quality of a particular state. We compare OEP to different MCTS variations that constrain the exploration to deal with the high branching factor in the turn-based multi-action game Hero Academy. While the constrained MCTS variations outperform the vanilla MCTS implementation by a large margin, OEP is able to search the space of plans more efficiently than any of the tested tree search methods as it has a relative advantage when the number of actions per turn increases.

    Original languageEnglish (US)
    JournalIEEE Transactions on Computational Intelligence and AI in Games
    DOIs
    StateAccepted/In press - Aug 10 2017

    Fingerprint

    Planning
    Monte Carlo methods

    Keywords

    • Evolutionary computation
    • Games
    • Monte Carlo methods
    • Planning
    • Real-time systems
    • Search problems

    ASJC Scopus subject areas

    • Control and Systems Engineering
    • Software
    • Artificial Intelligence
    • Electrical and Electronic Engineering

    Cite this

    Playing Multi-Action Adversarial Games : Online Evolutionary Planning versus Tree Search. / Justesen, Niels; Mahlmann, Tobias; Risi, Sebastian; Togelius, Julian.

    In: IEEE Transactions on Computational Intelligence and AI in Games, 10.08.2017.

    Research output: Contribution to journalArticle

    @article{13e7d03a6bed4f98aee5653d205c55ce,
    title = "Playing Multi-Action Adversarial Games: Online Evolutionary Planning versus Tree Search",
    abstract = "We address the problem of playing turn-based multi-action adversarial games, which include many strategy games with extremely high branching factors as players take multiple actions each turn. This leads to the breakdown of standard tree search methods, including Monte Carlo Tree Search (MCTS), as they become unable to reach a sufficient depth in the game tree. In this paper we introduce Online Evolutionary Planning (OEP) to address this challenge, which searches for combinations of actions to perform during a single turn guided by a fitness function that evaluates the quality of a particular state. We compare OEP to different MCTS variations that constrain the exploration to deal with the high branching factor in the turn-based multi-action game Hero Academy. While the constrained MCTS variations outperform the vanilla MCTS implementation by a large margin, OEP is able to search the space of plans more efficiently than any of the tested tree search methods as it has a relative advantage when the number of actions per turn increases.",
    keywords = "Evolutionary computation, Games, Monte Carlo methods, Planning, Real-time systems, Search problems",
    author = "Niels Justesen and Tobias Mahlmann and Sebastian Risi and Julian Togelius",
    year = "2017",
    month = "8",
    day = "10",
    doi = "10.1109/TCIAIG.2017.2738156",
    language = "English (US)",
    journal = "IEEE Transactions on Computational Intelligence and AI in Games",
    issn = "1943-068X",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - JOUR

    T1 - Playing Multi-Action Adversarial Games

    T2 - Online Evolutionary Planning versus Tree Search

    AU - Justesen, Niels

    AU - Mahlmann, Tobias

    AU - Risi, Sebastian

    AU - Togelius, Julian

    PY - 2017/8/10

    Y1 - 2017/8/10

    N2 - We address the problem of playing turn-based multi-action adversarial games, which include many strategy games with extremely high branching factors as players take multiple actions each turn. This leads to the breakdown of standard tree search methods, including Monte Carlo Tree Search (MCTS), as they become unable to reach a sufficient depth in the game tree. In this paper we introduce Online Evolutionary Planning (OEP) to address this challenge, which searches for combinations of actions to perform during a single turn guided by a fitness function that evaluates the quality of a particular state. We compare OEP to different MCTS variations that constrain the exploration to deal with the high branching factor in the turn-based multi-action game Hero Academy. While the constrained MCTS variations outperform the vanilla MCTS implementation by a large margin, OEP is able to search the space of plans more efficiently than any of the tested tree search methods as it has a relative advantage when the number of actions per turn increases.

    AB - We address the problem of playing turn-based multi-action adversarial games, which include many strategy games with extremely high branching factors as players take multiple actions each turn. This leads to the breakdown of standard tree search methods, including Monte Carlo Tree Search (MCTS), as they become unable to reach a sufficient depth in the game tree. In this paper we introduce Online Evolutionary Planning (OEP) to address this challenge, which searches for combinations of actions to perform during a single turn guided by a fitness function that evaluates the quality of a particular state. We compare OEP to different MCTS variations that constrain the exploration to deal with the high branching factor in the turn-based multi-action game Hero Academy. While the constrained MCTS variations outperform the vanilla MCTS implementation by a large margin, OEP is able to search the space of plans more efficiently than any of the tested tree search methods as it has a relative advantage when the number of actions per turn increases.

    KW - Evolutionary computation

    KW - Games

    KW - Monte Carlo methods

    KW - Planning

    KW - Real-time systems

    KW - Search problems

    UR - http://www.scopus.com/inward/record.url?scp=85029148021&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85029148021&partnerID=8YFLogxK

    U2 - 10.1109/TCIAIG.2017.2738156

    DO - 10.1109/TCIAIG.2017.2738156

    M3 - Article

    AN - SCOPUS:85029148021

    JO - IEEE Transactions on Computational Intelligence and AI in Games

    JF - IEEE Transactions on Computational Intelligence and AI in Games

    SN - 1943-068X

    ER -