Using graphics processors for high performance IR query processing

Shuai Ding, Jinru He, Hao Yan, Torsten Suel

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Web search engines are facing formidable performance challenges due to data sizes and query loads. The major engines have to process tens of thousands of queries per second over tens of billions of documents. To deal with this heavy work-load, such engines employ massively parallel systems consisting of thousands of machines. The significant cost of operating these systems has motivated a lot of recent research into more efficient query processing mechanisms. We investigate a new way to build such high performance IR systems using graphical processing units (GPUs). GPUs were originally designed to accelerate computer graphics applications through massive on-chip parallelism. Recently a number of researchers have studied how to use GPUs for other problem domains such as databases and scientific computing [9, 8, 12]. Our contribution here is to design a basic system architecture for GPU-based high-performance IR, to develop suitable algorithms for subtasks such as inverted list compression, list intersection, and top-κ scoring, and to show how to achieve highly efficient query processing on GPU-based systems. Our experimental results for a prototype GPU-based system on 25.2 million web pages shows promising gains in query throughput. Copyright is held by the International World Wide Web Conference Committee (IW3C2).

    Original languageEnglish (US)
    Title of host publicationWWW'09 - Proceedings of the 18th International World Wide Web Conference
    Pages421-430
    Number of pages10
    DOIs
    StatePublished - 2009
    Event18th International World Wide Web Conference, WWW 2009 - Madrid, Spain
    Duration: Apr 20 2009Apr 24 2009

    Other

    Other18th International World Wide Web Conference, WWW 2009
    CountrySpain
    CityMadrid
    Period4/20/094/24/09

    Fingerprint

    Query processing
    Processing
    Engines
    Natural sciences computing
    Computer graphics
    Search engines
    World Wide Web
    Websites
    Throughput
    Costs

    Keywords

    • GPU
    • Index compression
    • Query processing
    • Search engines

    ASJC Scopus subject areas

    • Computer Networks and Communications

    Cite this

    Ding, S., He, J., Yan, H., & Suel, T. (2009). Using graphics processors for high performance IR query processing. In WWW'09 - Proceedings of the 18th International World Wide Web Conference (pp. 421-430) https://doi.org/10.1145/1526709.1526766

    Using graphics processors for high performance IR query processing. / Ding, Shuai; He, Jinru; Yan, Hao; Suel, Torsten.

    WWW'09 - Proceedings of the 18th International World Wide Web Conference. 2009. p. 421-430.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Ding, S, He, J, Yan, H & Suel, T 2009, Using graphics processors for high performance IR query processing. in WWW'09 - Proceedings of the 18th International World Wide Web Conference. pp. 421-430, 18th International World Wide Web Conference, WWW 2009, Madrid, Spain, 4/20/09. https://doi.org/10.1145/1526709.1526766
    Ding S, He J, Yan H, Suel T. Using graphics processors for high performance IR query processing. In WWW'09 - Proceedings of the 18th International World Wide Web Conference. 2009. p. 421-430 https://doi.org/10.1145/1526709.1526766
    Ding, Shuai ; He, Jinru ; Yan, Hao ; Suel, Torsten. / Using graphics processors for high performance IR query processing. WWW'09 - Proceedings of the 18th International World Wide Web Conference. 2009. pp. 421-430
    @inproceedings{ed92a6853e114eacbb9dc7d72b0ae47e,
    title = "Using graphics processors for high performance IR query processing",
    abstract = "Web search engines are facing formidable performance challenges due to data sizes and query loads. The major engines have to process tens of thousands of queries per second over tens of billions of documents. To deal with this heavy work-load, such engines employ massively parallel systems consisting of thousands of machines. The significant cost of operating these systems has motivated a lot of recent research into more efficient query processing mechanisms. We investigate a new way to build such high performance IR systems using graphical processing units (GPUs). GPUs were originally designed to accelerate computer graphics applications through massive on-chip parallelism. Recently a number of researchers have studied how to use GPUs for other problem domains such as databases and scientific computing [9, 8, 12]. Our contribution here is to design a basic system architecture for GPU-based high-performance IR, to develop suitable algorithms for subtasks such as inverted list compression, list intersection, and top-κ scoring, and to show how to achieve highly efficient query processing on GPU-based systems. Our experimental results for a prototype GPU-based system on 25.2 million web pages shows promising gains in query throughput. Copyright is held by the International World Wide Web Conference Committee (IW3C2).",
    keywords = "GPU, Index compression, Query processing, Search engines",
    author = "Shuai Ding and Jinru He and Hao Yan and Torsten Suel",
    year = "2009",
    doi = "10.1145/1526709.1526766",
    language = "English (US)",
    isbn = "9781605584874",
    pages = "421--430",
    booktitle = "WWW'09 - Proceedings of the 18th International World Wide Web Conference",

    }

    TY - GEN

    T1 - Using graphics processors for high performance IR query processing

    AU - Ding, Shuai

    AU - He, Jinru

    AU - Yan, Hao

    AU - Suel, Torsten

    PY - 2009

    Y1 - 2009

    N2 - Web search engines are facing formidable performance challenges due to data sizes and query loads. The major engines have to process tens of thousands of queries per second over tens of billions of documents. To deal with this heavy work-load, such engines employ massively parallel systems consisting of thousands of machines. The significant cost of operating these systems has motivated a lot of recent research into more efficient query processing mechanisms. We investigate a new way to build such high performance IR systems using graphical processing units (GPUs). GPUs were originally designed to accelerate computer graphics applications through massive on-chip parallelism. Recently a number of researchers have studied how to use GPUs for other problem domains such as databases and scientific computing [9, 8, 12]. Our contribution here is to design a basic system architecture for GPU-based high-performance IR, to develop suitable algorithms for subtasks such as inverted list compression, list intersection, and top-κ scoring, and to show how to achieve highly efficient query processing on GPU-based systems. Our experimental results for a prototype GPU-based system on 25.2 million web pages shows promising gains in query throughput. Copyright is held by the International World Wide Web Conference Committee (IW3C2).

    AB - Web search engines are facing formidable performance challenges due to data sizes and query loads. The major engines have to process tens of thousands of queries per second over tens of billions of documents. To deal with this heavy work-load, such engines employ massively parallel systems consisting of thousands of machines. The significant cost of operating these systems has motivated a lot of recent research into more efficient query processing mechanisms. We investigate a new way to build such high performance IR systems using graphical processing units (GPUs). GPUs were originally designed to accelerate computer graphics applications through massive on-chip parallelism. Recently a number of researchers have studied how to use GPUs for other problem domains such as databases and scientific computing [9, 8, 12]. Our contribution here is to design a basic system architecture for GPU-based high-performance IR, to develop suitable algorithms for subtasks such as inverted list compression, list intersection, and top-κ scoring, and to show how to achieve highly efficient query processing on GPU-based systems. Our experimental results for a prototype GPU-based system on 25.2 million web pages shows promising gains in query throughput. Copyright is held by the International World Wide Web Conference Committee (IW3C2).

    KW - GPU

    KW - Index compression

    KW - Query processing

    KW - Search engines

    UR - http://www.scopus.com/inward/record.url?scp=77949822904&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=77949822904&partnerID=8YFLogxK

    U2 - 10.1145/1526709.1526766

    DO - 10.1145/1526709.1526766

    M3 - Conference contribution

    AN - SCOPUS:77949822904

    SN - 9781605584874

    SP - 421

    EP - 430

    BT - WWW'09 - Proceedings of the 18th International World Wide Web Conference

    ER -