Flow algorithms for parallel query optimization

Amol Deshpande, Lisa Hellerstein

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    We address the problem of minimizing the response time of a multi-way join query using pipelined (inter-operator) parallelism, in a parallel or a distributed environment. We observe that in order to fully exploit the parallelism in the system, we must consider a new class of " interleaving" plans, where multiple query plans are used simultaneously to minimize the response time of a query (or to maximize the tuple-throughput of the system). We cast the query planning problem in this environment as a "flow maximization problem", and present polynomial-time algorithms that (statically) And the optimal set of plans to use for a given query, for a large class of multi-way join queries. Our proposed algorithms also naturally extend to query optimization over web services. Finally we present an extensive experimental evaluation that demonstrates both the need to consider such plans in parallel query processing and the effectiveness of our algorithms.

    Original languageEnglish (US)
    Title of host publicationProceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08
    Pages754-763
    Number of pages10
    DOIs
    StatePublished - 2008
    Event2008 IEEE 24th International Conference on Data Engineering, ICDE'08 - Cancun, Mexico
    Duration: Apr 7 2008Apr 12 2008

    Other

    Other2008 IEEE 24th International Conference on Data Engineering, ICDE'08
    CountryMexico
    CityCancun
    Period4/7/084/12/08

    Fingerprint

    Query processing
    Web services
    Throughput
    Polynomials
    Planning

    ASJC Scopus subject areas

    • Information Systems
    • Signal Processing
    • Software

    Cite this

    Deshpande, A., & Hellerstein, L. (2008). Flow algorithms for parallel query optimization. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08 (pp. 754-763). [4497484] https://doi.org/10.1109/ICDE.2008.4497484

    Flow algorithms for parallel query optimization. / Deshpande, Amol; Hellerstein, Lisa.

    Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08. 2008. p. 754-763 4497484.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Deshpande, A & Hellerstein, L 2008, Flow algorithms for parallel query optimization. in Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08., 4497484, pp. 754-763, 2008 IEEE 24th International Conference on Data Engineering, ICDE'08, Cancun, Mexico, 4/7/08. https://doi.org/10.1109/ICDE.2008.4497484
    Deshpande A, Hellerstein L. Flow algorithms for parallel query optimization. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08. 2008. p. 754-763. 4497484 https://doi.org/10.1109/ICDE.2008.4497484
    Deshpande, Amol ; Hellerstein, Lisa. / Flow algorithms for parallel query optimization. Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08. 2008. pp. 754-763
    @inproceedings{9549cf8ade8d4f948083bf1cfebd6b3e,
    title = "Flow algorithms for parallel query optimization",
    abstract = "We address the problem of minimizing the response time of a multi-way join query using pipelined (inter-operator) parallelism, in a parallel or a distributed environment. We observe that in order to fully exploit the parallelism in the system, we must consider a new class of {"} interleaving{"} plans, where multiple query plans are used simultaneously to minimize the response time of a query (or to maximize the tuple-throughput of the system). We cast the query planning problem in this environment as a {"}flow maximization problem{"}, and present polynomial-time algorithms that (statically) And the optimal set of plans to use for a given query, for a large class of multi-way join queries. Our proposed algorithms also naturally extend to query optimization over web services. Finally we present an extensive experimental evaluation that demonstrates both the need to consider such plans in parallel query processing and the effectiveness of our algorithms.",
    author = "Amol Deshpande and Lisa Hellerstein",
    year = "2008",
    doi = "10.1109/ICDE.2008.4497484",
    language = "English (US)",
    isbn = "9781424418374",
    pages = "754--763",
    booktitle = "Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08",

    }

    TY - GEN

    T1 - Flow algorithms for parallel query optimization

    AU - Deshpande, Amol

    AU - Hellerstein, Lisa

    PY - 2008

    Y1 - 2008

    N2 - We address the problem of minimizing the response time of a multi-way join query using pipelined (inter-operator) parallelism, in a parallel or a distributed environment. We observe that in order to fully exploit the parallelism in the system, we must consider a new class of " interleaving" plans, where multiple query plans are used simultaneously to minimize the response time of a query (or to maximize the tuple-throughput of the system). We cast the query planning problem in this environment as a "flow maximization problem", and present polynomial-time algorithms that (statically) And the optimal set of plans to use for a given query, for a large class of multi-way join queries. Our proposed algorithms also naturally extend to query optimization over web services. Finally we present an extensive experimental evaluation that demonstrates both the need to consider such plans in parallel query processing and the effectiveness of our algorithms.

    AB - We address the problem of minimizing the response time of a multi-way join query using pipelined (inter-operator) parallelism, in a parallel or a distributed environment. We observe that in order to fully exploit the parallelism in the system, we must consider a new class of " interleaving" plans, where multiple query plans are used simultaneously to minimize the response time of a query (or to maximize the tuple-throughput of the system). We cast the query planning problem in this environment as a "flow maximization problem", and present polynomial-time algorithms that (statically) And the optimal set of plans to use for a given query, for a large class of multi-way join queries. Our proposed algorithms also naturally extend to query optimization over web services. Finally we present an extensive experimental evaluation that demonstrates both the need to consider such plans in parallel query processing and the effectiveness of our algorithms.

    UR - http://www.scopus.com/inward/record.url?scp=52649111582&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=52649111582&partnerID=8YFLogxK

    U2 - 10.1109/ICDE.2008.4497484

    DO - 10.1109/ICDE.2008.4497484

    M3 - Conference contribution

    AN - SCOPUS:52649111582

    SN - 9781424418374

    SP - 754

    EP - 763

    BT - Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08

    ER -