Deep Reinforcement Learning for General Video Game AI

Ruben Rodriguez Torrado, Philip Bontrager, Julian Togelius, Jialin Liu, Diego Perez-Liebana

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

    Original languageEnglish (US)
    Title of host publicationProceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018
    PublisherIEEE Computer Society
    Volume2018-August
    ISBN (Electronic)9781538643594
    DOIs
    StatePublished - Oct 11 2018
    Event14th IEEE Conference on Computational Intelligence and Games, CIG 2018 - Maastricht, Netherlands
    Duration: Aug 14 2018Aug 17 2018

    Other

    Other14th IEEE Conference on Computational Intelligence and Games, CIG 2018
    CountryNetherlands
    CityMaastricht
    Period8/14/188/17/18

    Fingerprint

    Reinforcement learning
    Benchmarking
    Learning algorithms
    Planning

    Keywords

    • Advantage actor critic
    • Deep Q-learning
    • Deep reinforcement learning
    • General video game AI
    • OpenAI Gym
    • Video game description language

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Graphics and Computer-Aided Design
    • Computer Vision and Pattern Recognition
    • Human-Computer Interaction
    • Software

    Cite this

    Torrado, R. R., Bontrager, P., Togelius, J., Liu, J., & Perez-Liebana, D. (2018). Deep Reinforcement Learning for General Video Game AI. In Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018 (Vol. 2018-August). [8490422] IEEE Computer Society. https://doi.org/10.1109/CIG.2018.8490422

    Deep Reinforcement Learning for General Video Game AI. / Torrado, Ruben Rodriguez; Bontrager, Philip; Togelius, Julian; Liu, Jialin; Perez-Liebana, Diego.

    Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. Vol. 2018-August IEEE Computer Society, 2018. 8490422.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Torrado, RR, Bontrager, P, Togelius, J, Liu, J & Perez-Liebana, D 2018, Deep Reinforcement Learning for General Video Game AI. in Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. vol. 2018-August, 8490422, IEEE Computer Society, 14th IEEE Conference on Computational Intelligence and Games, CIG 2018, Maastricht, Netherlands, 8/14/18. https://doi.org/10.1109/CIG.2018.8490422
    Torrado RR, Bontrager P, Togelius J, Liu J, Perez-Liebana D. Deep Reinforcement Learning for General Video Game AI. In Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. Vol. 2018-August. IEEE Computer Society. 2018. 8490422 https://doi.org/10.1109/CIG.2018.8490422
    Torrado, Ruben Rodriguez ; Bontrager, Philip ; Togelius, Julian ; Liu, Jialin ; Perez-Liebana, Diego. / Deep Reinforcement Learning for General Video Game AI. Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. Vol. 2018-August IEEE Computer Society, 2018.
    @inproceedings{4a31cc9c4bb7474ab25d8025799bd3cc,
    title = "Deep Reinforcement Learning for General Video Game AI",
    abstract = "The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.",
    keywords = "Advantage actor critic, Deep Q-learning, Deep reinforcement learning, General video game AI, OpenAI Gym, Video game description language",
    author = "Torrado, {Ruben Rodriguez} and Philip Bontrager and Julian Togelius and Jialin Liu and Diego Perez-Liebana",
    year = "2018",
    month = "10",
    day = "11",
    doi = "10.1109/CIG.2018.8490422",
    language = "English (US)",
    volume = "2018-August",
    booktitle = "Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018",
    publisher = "IEEE Computer Society",

    }

    TY - GEN

    T1 - Deep Reinforcement Learning for General Video Game AI

    AU - Torrado, Ruben Rodriguez

    AU - Bontrager, Philip

    AU - Togelius, Julian

    AU - Liu, Jialin

    AU - Perez-Liebana, Diego

    PY - 2018/10/11

    Y1 - 2018/10/11

    N2 - The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

    AB - The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

    KW - Advantage actor critic

    KW - Deep Q-learning

    KW - Deep reinforcement learning

    KW - General video game AI

    KW - OpenAI Gym

    KW - Video game description language

    UR - http://www.scopus.com/inward/record.url?scp=85056836621&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85056836621&partnerID=8YFLogxK

    U2 - 10.1109/CIG.2018.8490422

    DO - 10.1109/CIG.2018.8490422

    M3 - Conference contribution

    AN - SCOPUS:85056836621

    VL - 2018-August

    BT - Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018

    PB - IEEE Computer Society

    ER -