A scalable execution engine for package queries

Matteo Brucato, Azza Abouzied, Alexandra Meliou

    Research output: Contribution to journalArticle

    Abstract

    Many modern applications and real-world problems involve the design of item collections, or packages: from planning your daily meals all the way to mapping the universe. Despite the pervasive need for packages, traditional data management does not offer support for their definition and computation. This is because traditional database queries follow a powerful, but very simple model: a query defines constraints that each tuple in the result must satisfy. However, a system tasked with the design of packages cannot consider items independently; rather, the system needs to determine if a set of items collectively satisfy given criteria. In this paper, we present package queries, a new query model that extends traditional database queries to handle complex constraints and preferences over answer sets. We develop a full-fledged package query system, implemented on top of a traditional database engine. Our work makes several contributions. First, we design PaQL, a SQL-based query language that supports the declarative specification of package queries. Second, we present a fundamental strategy for evaluating package queries that combines the capabilities of databases and constraint optimization solvers. The core of our approach is a set of translation rules that transform a package query to an integer linear program. Third, we introduce an offline data partitioning strategy allowing query evaluation to scale to large data sizes. Fourth, we introduce SKETCHREFINE, an efficient and scalable algorithm for package evaluation, which offers strong approximation guarantees. Finally, we present extensive experiments over real-world data. Our results demonstrate that SKETCHREFINE is effective at deriving high-quality package results, and achieves runtime performance that is an order of magnitude faster than directly using ILP solvers over large datasets.

    Original languageEnglish (US)
    Pages (from-to)24-31
    Number of pages8
    JournalSIGMOD Record
    Volume46
    Issue number1
    DOIs
    StatePublished - Mar 1 2017

    Fingerprint

    Engines
    Inductive logic programming (ILP)
    Query languages
    Information management
    Specifications
    Planning
    Experiments

    ASJC Scopus subject areas

    • Software
    • Information Systems

    Cite this

    Brucato, M., Abouzied, A., & Meliou, A. (2017). A scalable execution engine for package queries. SIGMOD Record, 46(1), 24-31. https://doi.org/10.1145/3093754.3093761

    A scalable execution engine for package queries. / Brucato, Matteo; Abouzied, Azza; Meliou, Alexandra.

    In: SIGMOD Record, Vol. 46, No. 1, 01.03.2017, p. 24-31.

    Research output: Contribution to journalArticle

    Brucato, M, Abouzied, A & Meliou, A 2017, 'A scalable execution engine for package queries', SIGMOD Record, vol. 46, no. 1, pp. 24-31. https://doi.org/10.1145/3093754.3093761
    Brucato, Matteo ; Abouzied, Azza ; Meliou, Alexandra. / A scalable execution engine for package queries. In: SIGMOD Record. 2017 ; Vol. 46, No. 1. pp. 24-31.
    @article{64f5edd367ef4ba69f4e0893a67fae28,
    title = "A scalable execution engine for package queries",
    abstract = "Many modern applications and real-world problems involve the design of item collections, or packages: from planning your daily meals all the way to mapping the universe. Despite the pervasive need for packages, traditional data management does not offer support for their definition and computation. This is because traditional database queries follow a powerful, but very simple model: a query defines constraints that each tuple in the result must satisfy. However, a system tasked with the design of packages cannot consider items independently; rather, the system needs to determine if a set of items collectively satisfy given criteria. In this paper, we present package queries, a new query model that extends traditional database queries to handle complex constraints and preferences over answer sets. We develop a full-fledged package query system, implemented on top of a traditional database engine. Our work makes several contributions. First, we design PaQL, a SQL-based query language that supports the declarative specification of package queries. Second, we present a fundamental strategy for evaluating package queries that combines the capabilities of databases and constraint optimization solvers. The core of our approach is a set of translation rules that transform a package query to an integer linear program. Third, we introduce an offline data partitioning strategy allowing query evaluation to scale to large data sizes. Fourth, we introduce SKETCHREFINE, an efficient and scalable algorithm for package evaluation, which offers strong approximation guarantees. Finally, we present extensive experiments over real-world data. Our results demonstrate that SKETCHREFINE is effective at deriving high-quality package results, and achieves runtime performance that is an order of magnitude faster than directly using ILP solvers over large datasets.",
    author = "Matteo Brucato and Azza Abouzied and Alexandra Meliou",
    year = "2017",
    month = "3",
    day = "1",
    doi = "10.1145/3093754.3093761",
    language = "English (US)",
    volume = "46",
    pages = "24--31",
    journal = "SIGMOD Record",
    issn = "0163-5808",
    publisher = "Association for Computing Machinery (ACM)",
    number = "1",

    }

    TY - JOUR

    T1 - A scalable execution engine for package queries

    AU - Brucato, Matteo

    AU - Abouzied, Azza

    AU - Meliou, Alexandra

    PY - 2017/3/1

    Y1 - 2017/3/1

    N2 - Many modern applications and real-world problems involve the design of item collections, or packages: from planning your daily meals all the way to mapping the universe. Despite the pervasive need for packages, traditional data management does not offer support for their definition and computation. This is because traditional database queries follow a powerful, but very simple model: a query defines constraints that each tuple in the result must satisfy. However, a system tasked with the design of packages cannot consider items independently; rather, the system needs to determine if a set of items collectively satisfy given criteria. In this paper, we present package queries, a new query model that extends traditional database queries to handle complex constraints and preferences over answer sets. We develop a full-fledged package query system, implemented on top of a traditional database engine. Our work makes several contributions. First, we design PaQL, a SQL-based query language that supports the declarative specification of package queries. Second, we present a fundamental strategy for evaluating package queries that combines the capabilities of databases and constraint optimization solvers. The core of our approach is a set of translation rules that transform a package query to an integer linear program. Third, we introduce an offline data partitioning strategy allowing query evaluation to scale to large data sizes. Fourth, we introduce SKETCHREFINE, an efficient and scalable algorithm for package evaluation, which offers strong approximation guarantees. Finally, we present extensive experiments over real-world data. Our results demonstrate that SKETCHREFINE is effective at deriving high-quality package results, and achieves runtime performance that is an order of magnitude faster than directly using ILP solvers over large datasets.

    AB - Many modern applications and real-world problems involve the design of item collections, or packages: from planning your daily meals all the way to mapping the universe. Despite the pervasive need for packages, traditional data management does not offer support for their definition and computation. This is because traditional database queries follow a powerful, but very simple model: a query defines constraints that each tuple in the result must satisfy. However, a system tasked with the design of packages cannot consider items independently; rather, the system needs to determine if a set of items collectively satisfy given criteria. In this paper, we present package queries, a new query model that extends traditional database queries to handle complex constraints and preferences over answer sets. We develop a full-fledged package query system, implemented on top of a traditional database engine. Our work makes several contributions. First, we design PaQL, a SQL-based query language that supports the declarative specification of package queries. Second, we present a fundamental strategy for evaluating package queries that combines the capabilities of databases and constraint optimization solvers. The core of our approach is a set of translation rules that transform a package query to an integer linear program. Third, we introduce an offline data partitioning strategy allowing query evaluation to scale to large data sizes. Fourth, we introduce SKETCHREFINE, an efficient and scalable algorithm for package evaluation, which offers strong approximation guarantees. Finally, we present extensive experiments over real-world data. Our results demonstrate that SKETCHREFINE is effective at deriving high-quality package results, and achieves runtime performance that is an order of magnitude faster than directly using ILP solvers over large datasets.

    UR - http://www.scopus.com/inward/record.url?scp=85019865418&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85019865418&partnerID=8YFLogxK

    U2 - 10.1145/3093754.3093761

    DO - 10.1145/3093754.3093761

    M3 - Article

    VL - 46

    SP - 24

    EP - 31

    JO - SIGMOD Record

    JF - SIGMOD Record

    SN - 0163-5808

    IS - 1

    ER -