Super Mario evolution

Julian Togelius, Sergey Karakovskiy, Jan Koutník, Jürgen Schmidhuber

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    We introduce a new reinforcement learning benchmark based on the classic platform game Super Mario Bros. The benchmark has a high-dimensional input space, and achieving a good score requires sophisticated and varied strategies. However, it has tunable difficulty, and at the lowest difficulty setting decent score can be achieved using rudimentary strategies and a small fraction of the input space. To investigate the properties of the benchmark, we evolve neural network-based controllers using different network architectures and input spaces. We show that it is relatively easy to learn basic strategies capable of clearing individual levels of low difficulty, but that these controllers have problems with generalization to unseen levels and with taking larger parts of the input space into account. A number of directions worth exploring for learning better-performing strategies are discussed.

    Original languageEnglish (US)
    Title of host publicationCIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games
    Pages156-161
    Number of pages6
    DOIs
    StatePublished - 2009
    EventCIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games - Milano, Italy
    Duration: Sep 7 2009Sep 10 2009

    Other

    OtherCIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games
    CountryItaly
    CityMilano
    Period9/7/099/10/09

    Fingerprint

    Controllers
    Reinforcement learning
    Network architecture
    Neural networks

    Keywords

    • Input representation
    • Neuroevolution
    • Platform games
    • Super Mario Bros

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computational Theory and Mathematics
    • Computer Graphics and Computer-Aided Design
    • Human-Computer Interaction

    Cite this

    Togelius, J., Karakovskiy, S., Koutník, J., & Schmidhuber, J. (2009). Super Mario evolution. In CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games (pp. 156-161). [5286481] https://doi.org/10.1109/CIG.2009.5286481

    Super Mario evolution. / Togelius, Julian; Karakovskiy, Sergey; Koutník, Jan; Schmidhuber, Jürgen.

    CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games. 2009. p. 156-161 5286481.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Togelius, J, Karakovskiy, S, Koutník, J & Schmidhuber, J 2009, Super Mario evolution. in CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games., 5286481, pp. 156-161, CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games, Milano, Italy, 9/7/09. https://doi.org/10.1109/CIG.2009.5286481
    Togelius J, Karakovskiy S, Koutník J, Schmidhuber J. Super Mario evolution. In CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games. 2009. p. 156-161. 5286481 https://doi.org/10.1109/CIG.2009.5286481
    Togelius, Julian ; Karakovskiy, Sergey ; Koutník, Jan ; Schmidhuber, Jürgen. / Super Mario evolution. CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games. 2009. pp. 156-161
    @inproceedings{c2effabb71ff41e383c2069380adc03b,
    title = "Super Mario evolution",
    abstract = "We introduce a new reinforcement learning benchmark based on the classic platform game Super Mario Bros. The benchmark has a high-dimensional input space, and achieving a good score requires sophisticated and varied strategies. However, it has tunable difficulty, and at the lowest difficulty setting decent score can be achieved using rudimentary strategies and a small fraction of the input space. To investigate the properties of the benchmark, we evolve neural network-based controllers using different network architectures and input spaces. We show that it is relatively easy to learn basic strategies capable of clearing individual levels of low difficulty, but that these controllers have problems with generalization to unseen levels and with taking larger parts of the input space into account. A number of directions worth exploring for learning better-performing strategies are discussed.",
    keywords = "Input representation, Neuroevolution, Platform games, Super Mario Bros",
    author = "Julian Togelius and Sergey Karakovskiy and Jan Koutn{\'i}k and J{\"u}rgen Schmidhuber",
    year = "2009",
    doi = "10.1109/CIG.2009.5286481",
    language = "English (US)",
    isbn = "9781424448159",
    pages = "156--161",
    booktitle = "CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games",

    }

    TY - GEN

    T1 - Super Mario evolution

    AU - Togelius, Julian

    AU - Karakovskiy, Sergey

    AU - Koutník, Jan

    AU - Schmidhuber, Jürgen

    PY - 2009

    Y1 - 2009

    N2 - We introduce a new reinforcement learning benchmark based on the classic platform game Super Mario Bros. The benchmark has a high-dimensional input space, and achieving a good score requires sophisticated and varied strategies. However, it has tunable difficulty, and at the lowest difficulty setting decent score can be achieved using rudimentary strategies and a small fraction of the input space. To investigate the properties of the benchmark, we evolve neural network-based controllers using different network architectures and input spaces. We show that it is relatively easy to learn basic strategies capable of clearing individual levels of low difficulty, but that these controllers have problems with generalization to unseen levels and with taking larger parts of the input space into account. A number of directions worth exploring for learning better-performing strategies are discussed.

    AB - We introduce a new reinforcement learning benchmark based on the classic platform game Super Mario Bros. The benchmark has a high-dimensional input space, and achieving a good score requires sophisticated and varied strategies. However, it has tunable difficulty, and at the lowest difficulty setting decent score can be achieved using rudimentary strategies and a small fraction of the input space. To investigate the properties of the benchmark, we evolve neural network-based controllers using different network architectures and input spaces. We show that it is relatively easy to learn basic strategies capable of clearing individual levels of low difficulty, but that these controllers have problems with generalization to unseen levels and with taking larger parts of the input space into account. A number of directions worth exploring for learning better-performing strategies are discussed.

    KW - Input representation

    KW - Neuroevolution

    KW - Platform games

    KW - Super Mario Bros

    UR - http://www.scopus.com/inward/record.url?scp=71549138688&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=71549138688&partnerID=8YFLogxK

    U2 - 10.1109/CIG.2009.5286481

    DO - 10.1109/CIG.2009.5286481

    M3 - Conference contribution

    AN - SCOPUS:71549138688

    SN - 9781424448159

    SP - 156

    EP - 161

    BT - CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games

    ER -