Analyzing the robustness of general video game playing agents

Diego Perez-Liebana, Spyridon Samothrakis, Julian Togelius, Tom Schaul, Simon M. Lucas

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    This paper presents a study on the robustness and variability of performance of general video game-playing agents. Agents analyzed includes those that won the different legs of the 2014 and 2015 General Video Game AI Competitions, and two sample agents distributed with its framework. Initially, these agents are run in four games and ranked according to the rules of the competition. Then, different modifications to the reward signal of the games are proposed and noise is introduced in either the actions executed by the controller, their forward model, or both. Results show that it is possible to produce a significant change in the rankings by introducing the modifications proposed here. This is an important result because it enables the set of human-authored games to be automatically expanded by adding parameter-varied versions that add information and insight into the relative strengths of the agents under test. Results also show that some controllers perform well under almost all conditions, a testament to the robustness of the GVGAI benchmark.

    Original languageEnglish (US)
    Title of host publication2016 IEEE Conference on Computational Intelligence and Games, CIG 2016
    PublisherIEEE Computer Society
    ISBN (Electronic)9781509018833
    DOIs
    StatePublished - Feb 21 2017
    Event2016 IEEE Conference on Computational Intelligence and Games, CIG 2016 - Santorini, Greece
    Duration: Sep 20 2016Sep 23 2016

    Other

    Other2016 IEEE Conference on Computational Intelligence and Games, CIG 2016
    CountryGreece
    CitySantorini
    Period9/20/169/23/16

    Fingerprint

    Controllers

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Graphics and Computer-Aided Design
    • Computer Vision and Pattern Recognition
    • Human-Computer Interaction
    • Software

    Cite this

    Perez-Liebana, D., Samothrakis, S., Togelius, J., Schaul, T., & Lucas, S. M. (2017). Analyzing the robustness of general video game playing agents. In 2016 IEEE Conference on Computational Intelligence and Games, CIG 2016 [7860430] IEEE Computer Society. https://doi.org/10.1109/CIG.2016.7860430

    Analyzing the robustness of general video game playing agents. / Perez-Liebana, Diego; Samothrakis, Spyridon; Togelius, Julian; Schaul, Tom; Lucas, Simon M.

    2016 IEEE Conference on Computational Intelligence and Games, CIG 2016. IEEE Computer Society, 2017. 7860430.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Perez-Liebana, D, Samothrakis, S, Togelius, J, Schaul, T & Lucas, SM 2017, Analyzing the robustness of general video game playing agents. in 2016 IEEE Conference on Computational Intelligence and Games, CIG 2016., 7860430, IEEE Computer Society, 2016 IEEE Conference on Computational Intelligence and Games, CIG 2016, Santorini, Greece, 9/20/16. https://doi.org/10.1109/CIG.2016.7860430
    Perez-Liebana D, Samothrakis S, Togelius J, Schaul T, Lucas SM. Analyzing the robustness of general video game playing agents. In 2016 IEEE Conference on Computational Intelligence and Games, CIG 2016. IEEE Computer Society. 2017. 7860430 https://doi.org/10.1109/CIG.2016.7860430
    Perez-Liebana, Diego ; Samothrakis, Spyridon ; Togelius, Julian ; Schaul, Tom ; Lucas, Simon M. / Analyzing the robustness of general video game playing agents. 2016 IEEE Conference on Computational Intelligence and Games, CIG 2016. IEEE Computer Society, 2017.
    @inproceedings{7ab7a10d905d4be1b71c3c40a26e6e4c,
    title = "Analyzing the robustness of general video game playing agents",
    abstract = "This paper presents a study on the robustness and variability of performance of general video game-playing agents. Agents analyzed includes those that won the different legs of the 2014 and 2015 General Video Game AI Competitions, and two sample agents distributed with its framework. Initially, these agents are run in four games and ranked according to the rules of the competition. Then, different modifications to the reward signal of the games are proposed and noise is introduced in either the actions executed by the controller, their forward model, or both. Results show that it is possible to produce a significant change in the rankings by introducing the modifications proposed here. This is an important result because it enables the set of human-authored games to be automatically expanded by adding parameter-varied versions that add information and insight into the relative strengths of the agents under test. Results also show that some controllers perform well under almost all conditions, a testament to the robustness of the GVGAI benchmark.",
    author = "Diego Perez-Liebana and Spyridon Samothrakis and Julian Togelius and Tom Schaul and Lucas, {Simon M.}",
    year = "2017",
    month = "2",
    day = "21",
    doi = "10.1109/CIG.2016.7860430",
    language = "English (US)",
    booktitle = "2016 IEEE Conference on Computational Intelligence and Games, CIG 2016",
    publisher = "IEEE Computer Society",
    address = "United States",

    }

    TY - GEN

    T1 - Analyzing the robustness of general video game playing agents

    AU - Perez-Liebana, Diego

    AU - Samothrakis, Spyridon

    AU - Togelius, Julian

    AU - Schaul, Tom

    AU - Lucas, Simon M.

    PY - 2017/2/21

    Y1 - 2017/2/21

    N2 - This paper presents a study on the robustness and variability of performance of general video game-playing agents. Agents analyzed includes those that won the different legs of the 2014 and 2015 General Video Game AI Competitions, and two sample agents distributed with its framework. Initially, these agents are run in four games and ranked according to the rules of the competition. Then, different modifications to the reward signal of the games are proposed and noise is introduced in either the actions executed by the controller, their forward model, or both. Results show that it is possible to produce a significant change in the rankings by introducing the modifications proposed here. This is an important result because it enables the set of human-authored games to be automatically expanded by adding parameter-varied versions that add information and insight into the relative strengths of the agents under test. Results also show that some controllers perform well under almost all conditions, a testament to the robustness of the GVGAI benchmark.

    AB - This paper presents a study on the robustness and variability of performance of general video game-playing agents. Agents analyzed includes those that won the different legs of the 2014 and 2015 General Video Game AI Competitions, and two sample agents distributed with its framework. Initially, these agents are run in four games and ranked according to the rules of the competition. Then, different modifications to the reward signal of the games are proposed and noise is introduced in either the actions executed by the controller, their forward model, or both. Results show that it is possible to produce a significant change in the rankings by introducing the modifications proposed here. This is an important result because it enables the set of human-authored games to be automatically expanded by adding parameter-varied versions that add information and insight into the relative strengths of the agents under test. Results also show that some controllers perform well under almost all conditions, a testament to the robustness of the GVGAI benchmark.

    UR - http://www.scopus.com/inward/record.url?scp=85015399275&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85015399275&partnerID=8YFLogxK

    U2 - 10.1109/CIG.2016.7860430

    DO - 10.1109/CIG.2016.7860430

    M3 - Conference contribution

    AN - SCOPUS:85015399275

    BT - 2016 IEEE Conference on Computational Intelligence and Games, CIG 2016

    PB - IEEE Computer Society

    ER -