How to Run a Successful Game-Based AI Competition

Julian Togelius

    Research output: Contribution to journalArticle

    Abstract

    Game-based competitions are commonly used within the Computational Intelligence (CI) and Artificial Intelligence (AI) in games community to benchmark algorithms and to attract new researchers. While many competitions have been organized based on different games, the success of these competitions is highly varied. This short paper is a self-help paper for competition organizers and aspiring competition organizers. After analyzing the fate of a number of recent competitions, some factors likely to contribute to the success or failure of a competition are laid out, and a set of concrete recommendations is offered. There is also a discussion of how to write up game-based AI competitions and what we can ultimately learn from them.

    Original languageEnglish (US)
    Article number6937175
    Pages (from-to)95-100
    Number of pages6
    JournalIEEE Transactions on Computational Intelligence and AI in Games
    Volume8
    Issue number1
    DOIs
    StatePublished - Mar 1 2016

    Fingerprint

    Artificial intelligence

    Keywords

    • Benchmarking
    • competitions

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Software
    • Control and Systems Engineering
    • Electrical and Electronic Engineering

    Cite this

    How to Run a Successful Game-Based AI Competition. / Togelius, Julian.

    In: IEEE Transactions on Computational Intelligence and AI in Games, Vol. 8, No. 1, 6937175, 01.03.2016, p. 95-100.

    Research output: Contribution to journalArticle

    @article{11dba86982ae47ecac626399e8673c3f,
    title = "How to Run a Successful Game-Based AI Competition",
    abstract = "Game-based competitions are commonly used within the Computational Intelligence (CI) and Artificial Intelligence (AI) in games community to benchmark algorithms and to attract new researchers. While many competitions have been organized based on different games, the success of these competitions is highly varied. This short paper is a self-help paper for competition organizers and aspiring competition organizers. After analyzing the fate of a number of recent competitions, some factors likely to contribute to the success or failure of a competition are laid out, and a set of concrete recommendations is offered. There is also a discussion of how to write up game-based AI competitions and what we can ultimately learn from them.",
    keywords = "Benchmarking, competitions",
    author = "Julian Togelius",
    year = "2016",
    month = "3",
    day = "1",
    doi = "10.1109/TCIAIG.2014.2365470",
    language = "English (US)",
    volume = "8",
    pages = "95--100",
    journal = "IEEE Transactions on Computational Intelligence and AI in Games",
    issn = "1943-068X",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    number = "1",

    }

    TY - JOUR

    T1 - How to Run a Successful Game-Based AI Competition

    AU - Togelius, Julian

    PY - 2016/3/1

    Y1 - 2016/3/1

    N2 - Game-based competitions are commonly used within the Computational Intelligence (CI) and Artificial Intelligence (AI) in games community to benchmark algorithms and to attract new researchers. While many competitions have been organized based on different games, the success of these competitions is highly varied. This short paper is a self-help paper for competition organizers and aspiring competition organizers. After analyzing the fate of a number of recent competitions, some factors likely to contribute to the success or failure of a competition are laid out, and a set of concrete recommendations is offered. There is also a discussion of how to write up game-based AI competitions and what we can ultimately learn from them.

    AB - Game-based competitions are commonly used within the Computational Intelligence (CI) and Artificial Intelligence (AI) in games community to benchmark algorithms and to attract new researchers. While many competitions have been organized based on different games, the success of these competitions is highly varied. This short paper is a self-help paper for competition organizers and aspiring competition organizers. After analyzing the fate of a number of recent competitions, some factors likely to contribute to the success or failure of a competition are laid out, and a set of concrete recommendations is offered. There is also a discussion of how to write up game-based AI competitions and what we can ultimately learn from them.

    KW - Benchmarking

    KW - competitions

    UR - http://www.scopus.com/inward/record.url?scp=84963716398&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84963716398&partnerID=8YFLogxK

    U2 - 10.1109/TCIAIG.2014.2365470

    DO - 10.1109/TCIAIG.2014.2365470

    M3 - Article

    VL - 8

    SP - 95

    EP - 100

    JO - IEEE Transactions on Computational Intelligence and AI in Games

    JF - IEEE Transactions on Computational Intelligence and AI in Games

    SN - 1943-068X

    IS - 1

    M1 - 6937175

    ER -