Improved techniques for result caching in Web search engines

Qingqing Gan, Torsten Suel

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Query processing is a major cost factor in operating large web search engines. In this paper, we study query result caching, one of the main techniques used to optimize query processing performance. Our first contribution is a study of result caching as a weighted caching problem. Most previous work has focused on optimizing cache hit ratios, but given that processing costs of queries can vary very significantly we argue that total cost savings also need to be considered. We describe and evaluate several algorithms for weighted result caching, and study the impact of Zipf-based query distributions on result caching. Our second and main contribution is a new set of feature-based cache eviction policies that achieve significant improvements over all previous methods, substantially narrowing the existing performance gap to the theoretically optimal (clairvoyant) method. Finally, using the same approach, we also obtain performance gains for the related problem of inverted list caching. Copyright is held by the International World Wide Web Conference Committee (IW3C2).

    Original languageEnglish (US)
    Title of host publicationWWW'09 - Proceedings of the 18th International World Wide Web Conference
    Pages431-440
    Number of pages10
    DOIs
    StatePublished - 2009
    Event18th International World Wide Web Conference, WWW 2009 - Madrid, Spain
    Duration: Apr 20 2009Apr 24 2009

    Other

    Other18th International World Wide Web Conference, WWW 2009
    CountrySpain
    CityMadrid
    Period4/20/094/24/09

    Fingerprint

    Search engines
    Query processing
    Costs
    World Wide Web
    Processing

    Keywords

    • Index caching
    • Result caching
    • Search engines
    • Weighted caching

    ASJC Scopus subject areas

    • Computer Networks and Communications

    Cite this

    Gan, Q., & Suel, T. (2009). Improved techniques for result caching in Web search engines. In WWW'09 - Proceedings of the 18th International World Wide Web Conference (pp. 431-440) https://doi.org/10.1145/1526709.1526768

    Improved techniques for result caching in Web search engines. / Gan, Qingqing; Suel, Torsten.

    WWW'09 - Proceedings of the 18th International World Wide Web Conference. 2009. p. 431-440.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Gan, Q & Suel, T 2009, Improved techniques for result caching in Web search engines. in WWW'09 - Proceedings of the 18th International World Wide Web Conference. pp. 431-440, 18th International World Wide Web Conference, WWW 2009, Madrid, Spain, 4/20/09. https://doi.org/10.1145/1526709.1526768
    Gan Q, Suel T. Improved techniques for result caching in Web search engines. In WWW'09 - Proceedings of the 18th International World Wide Web Conference. 2009. p. 431-440 https://doi.org/10.1145/1526709.1526768
    Gan, Qingqing ; Suel, Torsten. / Improved techniques for result caching in Web search engines. WWW'09 - Proceedings of the 18th International World Wide Web Conference. 2009. pp. 431-440
    @inproceedings{089505ef543445f184d0af6cd7eb7f89,
    title = "Improved techniques for result caching in Web search engines",
    abstract = "Query processing is a major cost factor in operating large web search engines. In this paper, we study query result caching, one of the main techniques used to optimize query processing performance. Our first contribution is a study of result caching as a weighted caching problem. Most previous work has focused on optimizing cache hit ratios, but given that processing costs of queries can vary very significantly we argue that total cost savings also need to be considered. We describe and evaluate several algorithms for weighted result caching, and study the impact of Zipf-based query distributions on result caching. Our second and main contribution is a new set of feature-based cache eviction policies that achieve significant improvements over all previous methods, substantially narrowing the existing performance gap to the theoretically optimal (clairvoyant) method. Finally, using the same approach, we also obtain performance gains for the related problem of inverted list caching. Copyright is held by the International World Wide Web Conference Committee (IW3C2).",
    keywords = "Index caching, Result caching, Search engines, Weighted caching",
    author = "Qingqing Gan and Torsten Suel",
    year = "2009",
    doi = "10.1145/1526709.1526768",
    language = "English (US)",
    isbn = "9781605584874",
    pages = "431--440",
    booktitle = "WWW'09 - Proceedings of the 18th International World Wide Web Conference",

    }

    TY - GEN

    T1 - Improved techniques for result caching in Web search engines

    AU - Gan, Qingqing

    AU - Suel, Torsten

    PY - 2009

    Y1 - 2009

    N2 - Query processing is a major cost factor in operating large web search engines. In this paper, we study query result caching, one of the main techniques used to optimize query processing performance. Our first contribution is a study of result caching as a weighted caching problem. Most previous work has focused on optimizing cache hit ratios, but given that processing costs of queries can vary very significantly we argue that total cost savings also need to be considered. We describe and evaluate several algorithms for weighted result caching, and study the impact of Zipf-based query distributions on result caching. Our second and main contribution is a new set of feature-based cache eviction policies that achieve significant improvements over all previous methods, substantially narrowing the existing performance gap to the theoretically optimal (clairvoyant) method. Finally, using the same approach, we also obtain performance gains for the related problem of inverted list caching. Copyright is held by the International World Wide Web Conference Committee (IW3C2).

    AB - Query processing is a major cost factor in operating large web search engines. In this paper, we study query result caching, one of the main techniques used to optimize query processing performance. Our first contribution is a study of result caching as a weighted caching problem. Most previous work has focused on optimizing cache hit ratios, but given that processing costs of queries can vary very significantly we argue that total cost savings also need to be considered. We describe and evaluate several algorithms for weighted result caching, and study the impact of Zipf-based query distributions on result caching. Our second and main contribution is a new set of feature-based cache eviction policies that achieve significant improvements over all previous methods, substantially narrowing the existing performance gap to the theoretically optimal (clairvoyant) method. Finally, using the same approach, we also obtain performance gains for the related problem of inverted list caching. Copyright is held by the International World Wide Web Conference Committee (IW3C2).

    KW - Index caching

    KW - Result caching

    KW - Search engines

    KW - Weighted caching

    UR - http://www.scopus.com/inward/record.url?scp=70350616726&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=70350616726&partnerID=8YFLogxK

    U2 - 10.1145/1526709.1526768

    DO - 10.1145/1526709.1526768

    M3 - Conference contribution

    SN - 9781605584874

    SP - 431

    EP - 440

    BT - WWW'09 - Proceedings of the 18th International World Wide Web Conference

    ER -