Hierarchical controller learning in a first-person shooter

Niels Van Hoorn, Julian Togelius, Jürgen Schmidhuber

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    We describe the architecture of a hierarchical learning-based controller for bots in the First-Person Shooter (FPS) game Unreal Tournament 2004. The controller is inspired by the subsumption architecture commonly used in behaviour-based robotics. A behaviour selector decides which of three sub-controllers gets to control the bot at each time step. Each controller is implemented as a recurrent neural network, and trained with artificial evolution to perform respectively combat, exploration and path following. The behaviour selector is trained with a multiobjective evolutionary algorithm to achieve an effective balancing of the lower-level behaviours. We argue that FPS games provide good environments for studying the learning of complex behaviours, and that the methods proposed here can help developing interesting opponents for games.

    Original languageEnglish (US)
    Title of host publicationCIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games
    Pages294-301
    Number of pages8
    DOIs
    StatePublished - 2009
    EventCIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games - Milano, Italy
    Duration: Sep 7 2009Sep 10 2009

    Other

    OtherCIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games
    CountryItaly
    CityMilano
    Period9/7/099/10/09

    Fingerprint

    Controllers
    Recurrent neural networks
    Evolutionary algorithms
    Robotics

    Keywords

    • Action selection
    • Behaviour-based robotics
    • Evolutionary algorithms
    • First-person shooters
    • FPS
    • Neural networks
    • Subsumption architecture

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computational Theory and Mathematics
    • Computer Graphics and Computer-Aided Design
    • Human-Computer Interaction

    Cite this

    Van Hoorn, N., Togelius, J., & Schmidhuber, J. (2009). Hierarchical controller learning in a first-person shooter. In CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games (pp. 294-301). [5286463] https://doi.org/10.1109/CIG.2009.5286463

    Hierarchical controller learning in a first-person shooter. / Van Hoorn, Niels; Togelius, Julian; Schmidhuber, Jürgen.

    CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games. 2009. p. 294-301 5286463.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Van Hoorn, N, Togelius, J & Schmidhuber, J 2009, Hierarchical controller learning in a first-person shooter. in CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games., 5286463, pp. 294-301, CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games, Milano, Italy, 9/7/09. https://doi.org/10.1109/CIG.2009.5286463
    Van Hoorn N, Togelius J, Schmidhuber J. Hierarchical controller learning in a first-person shooter. In CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games. 2009. p. 294-301. 5286463 https://doi.org/10.1109/CIG.2009.5286463
    Van Hoorn, Niels ; Togelius, Julian ; Schmidhuber, Jürgen. / Hierarchical controller learning in a first-person shooter. CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games. 2009. pp. 294-301
    @inproceedings{2ab51c23107d491abc319a3fe6dad33f,
    title = "Hierarchical controller learning in a first-person shooter",
    abstract = "We describe the architecture of a hierarchical learning-based controller for bots in the First-Person Shooter (FPS) game Unreal Tournament 2004. The controller is inspired by the subsumption architecture commonly used in behaviour-based robotics. A behaviour selector decides which of three sub-controllers gets to control the bot at each time step. Each controller is implemented as a recurrent neural network, and trained with artificial evolution to perform respectively combat, exploration and path following. The behaviour selector is trained with a multiobjective evolutionary algorithm to achieve an effective balancing of the lower-level behaviours. We argue that FPS games provide good environments for studying the learning of complex behaviours, and that the methods proposed here can help developing interesting opponents for games.",
    keywords = "Action selection, Behaviour-based robotics, Evolutionary algorithms, First-person shooters, FPS, Neural networks, Subsumption architecture",
    author = "{Van Hoorn}, Niels and Julian Togelius and J{\"u}rgen Schmidhuber",
    year = "2009",
    doi = "10.1109/CIG.2009.5286463",
    language = "English (US)",
    isbn = "9781424448159",
    pages = "294--301",
    booktitle = "CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games",

    }

    TY - GEN

    T1 - Hierarchical controller learning in a first-person shooter

    AU - Van Hoorn, Niels

    AU - Togelius, Julian

    AU - Schmidhuber, Jürgen

    PY - 2009

    Y1 - 2009

    N2 - We describe the architecture of a hierarchical learning-based controller for bots in the First-Person Shooter (FPS) game Unreal Tournament 2004. The controller is inspired by the subsumption architecture commonly used in behaviour-based robotics. A behaviour selector decides which of three sub-controllers gets to control the bot at each time step. Each controller is implemented as a recurrent neural network, and trained with artificial evolution to perform respectively combat, exploration and path following. The behaviour selector is trained with a multiobjective evolutionary algorithm to achieve an effective balancing of the lower-level behaviours. We argue that FPS games provide good environments for studying the learning of complex behaviours, and that the methods proposed here can help developing interesting opponents for games.

    AB - We describe the architecture of a hierarchical learning-based controller for bots in the First-Person Shooter (FPS) game Unreal Tournament 2004. The controller is inspired by the subsumption architecture commonly used in behaviour-based robotics. A behaviour selector decides which of three sub-controllers gets to control the bot at each time step. Each controller is implemented as a recurrent neural network, and trained with artificial evolution to perform respectively combat, exploration and path following. The behaviour selector is trained with a multiobjective evolutionary algorithm to achieve an effective balancing of the lower-level behaviours. We argue that FPS games provide good environments for studying the learning of complex behaviours, and that the methods proposed here can help developing interesting opponents for games.

    KW - Action selection

    KW - Behaviour-based robotics

    KW - Evolutionary algorithms

    KW - First-person shooters

    KW - FPS

    KW - Neural networks

    KW - Subsumption architecture

    UR - http://www.scopus.com/inward/record.url?scp=71549132227&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=71549132227&partnerID=8YFLogxK

    U2 - 10.1109/CIG.2009.5286463

    DO - 10.1109/CIG.2009.5286463

    M3 - Conference contribution

    AN - SCOPUS:71549132227

    SN - 9781424448159

    SP - 294

    EP - 301

    BT - CIG2009 - 2009 IEEE Symposium on Computational Intelligence and Games

    ER -