Package queries: efficient and scalable computation of high-order constraints

Matteo Brucato, Azza Abouzied, Alexandra Meliou

Research output: Contribution to journalArticle

Abstract

Traditional database queries follow a simple model: they define constraints that each tuple in the result must satisfy. This model is computationally efficient, as the database system can evaluate the query conditions on each tuple individually. However, many practical, real-world problems require a collection of result tuples to satisfy constraints collectively, rather than individually. In this paper, we present package queries, a new query model that extends traditional database queries to handle complex constraints and preferences over answer sets. We develop a full-fledged package query system, implemented on top of a traditional database engine. Our work makes several contributions. (1) We design PaQL, a SQL-based query language that supports the declarative specification of package queries. We prove that PaQL is at least as expressive as integer linear programming, and therefore, evaluation of package queries is NP-hard. (2) We present a fundamental evaluation strategy that combines the capabilities of databases and constraint optimization solvers to derive solutions to package queries. The core of our approach is a set of translation rules that transform a package query to an integer linear program. (3) We introduce an offline data partitioning strategy allowing query evaluation to scale to large data sizes. (4) We introduce SketchRefine, a scalable algorithm for package evaluation, with strong approximation guarantees [(Formula presented.)-factor approximation]. (5) We present a method for parallelizing the Refine phase of SketchRefine. (6) We present an empirical study of the efficiency gains of providing integer solvers with starting solutions. (7) We present extensive experiments over real-world and benchmark data. The results demonstrate that our methods are effective at deriving high-quality package results and achieve runtime performance that is an order of magnitude faster than directly using ILP solvers over large datasets.

Original languageEnglish (US)
Pages (from-to)1-26
Number of pages26
JournalVLDB Journal
DOIs
StateAccepted/In press - Oct 24 2017

Fingerprint

Inductive logic programming (ILP)
Query languages
Linear programming
Engines
Specifications
Experiments

Keywords

  • Approximation algorithm
  • Integer linear programming
  • Package queries
  • PaQL
  • SketchRefine

ASJC Scopus subject areas

  • Information Systems
  • Hardware and Architecture

Cite this

Package queries : efficient and scalable computation of high-order constraints. / Brucato, Matteo; Abouzied, Azza; Meliou, Alexandra.

In: VLDB Journal, 24.10.2017, p. 1-26.

Research output: Contribution to journalArticle

@article{2fdc342a3ea248be9562bf7571f135a6,
title = "Package queries: efficient and scalable computation of high-order constraints",
abstract = "Traditional database queries follow a simple model: they define constraints that each tuple in the result must satisfy. This model is computationally efficient, as the database system can evaluate the query conditions on each tuple individually. However, many practical, real-world problems require a collection of result tuples to satisfy constraints collectively, rather than individually. In this paper, we present package queries, a new query model that extends traditional database queries to handle complex constraints and preferences over answer sets. We develop a full-fledged package query system, implemented on top of a traditional database engine. Our work makes several contributions. (1) We design PaQL, a SQL-based query language that supports the declarative specification of package queries. We prove that PaQL is at least as expressive as integer linear programming, and therefore, evaluation of package queries is NP-hard. (2) We present a fundamental evaluation strategy that combines the capabilities of databases and constraint optimization solvers to derive solutions to package queries. The core of our approach is a set of translation rules that transform a package query to an integer linear program. (3) We introduce an offline data partitioning strategy allowing query evaluation to scale to large data sizes. (4) We introduce SketchRefine, a scalable algorithm for package evaluation, with strong approximation guarantees [(Formula presented.)-factor approximation]. (5) We present a method for parallelizing the Refine phase of SketchRefine. (6) We present an empirical study of the efficiency gains of providing integer solvers with starting solutions. (7) We present extensive experiments over real-world and benchmark data. The results demonstrate that our methods are effective at deriving high-quality package results and achieve runtime performance that is an order of magnitude faster than directly using ILP solvers over large datasets.",
keywords = "Approximation algorithm, Integer linear programming, Package queries, PaQL, SketchRefine",
author = "Matteo Brucato and Azza Abouzied and Alexandra Meliou",
year = "2017",
month = "10",
day = "24",
doi = "10.1007/s00778-017-0483-4",
language = "English (US)",
pages = "1--26",
journal = "VLDB Journal",
issn = "1066-8888",
publisher = "Springer New York",

}

TY - JOUR

T1 - Package queries

T2 - efficient and scalable computation of high-order constraints

AU - Brucato, Matteo

AU - Abouzied, Azza

AU - Meliou, Alexandra

PY - 2017/10/24

Y1 - 2017/10/24

N2 - Traditional database queries follow a simple model: they define constraints that each tuple in the result must satisfy. This model is computationally efficient, as the database system can evaluate the query conditions on each tuple individually. However, many practical, real-world problems require a collection of result tuples to satisfy constraints collectively, rather than individually. In this paper, we present package queries, a new query model that extends traditional database queries to handle complex constraints and preferences over answer sets. We develop a full-fledged package query system, implemented on top of a traditional database engine. Our work makes several contributions. (1) We design PaQL, a SQL-based query language that supports the declarative specification of package queries. We prove that PaQL is at least as expressive as integer linear programming, and therefore, evaluation of package queries is NP-hard. (2) We present a fundamental evaluation strategy that combines the capabilities of databases and constraint optimization solvers to derive solutions to package queries. The core of our approach is a set of translation rules that transform a package query to an integer linear program. (3) We introduce an offline data partitioning strategy allowing query evaluation to scale to large data sizes. (4) We introduce SketchRefine, a scalable algorithm for package evaluation, with strong approximation guarantees [(Formula presented.)-factor approximation]. (5) We present a method for parallelizing the Refine phase of SketchRefine. (6) We present an empirical study of the efficiency gains of providing integer solvers with starting solutions. (7) We present extensive experiments over real-world and benchmark data. The results demonstrate that our methods are effective at deriving high-quality package results and achieve runtime performance that is an order of magnitude faster than directly using ILP solvers over large datasets.

AB - Traditional database queries follow a simple model: they define constraints that each tuple in the result must satisfy. This model is computationally efficient, as the database system can evaluate the query conditions on each tuple individually. However, many practical, real-world problems require a collection of result tuples to satisfy constraints collectively, rather than individually. In this paper, we present package queries, a new query model that extends traditional database queries to handle complex constraints and preferences over answer sets. We develop a full-fledged package query system, implemented on top of a traditional database engine. Our work makes several contributions. (1) We design PaQL, a SQL-based query language that supports the declarative specification of package queries. We prove that PaQL is at least as expressive as integer linear programming, and therefore, evaluation of package queries is NP-hard. (2) We present a fundamental evaluation strategy that combines the capabilities of databases and constraint optimization solvers to derive solutions to package queries. The core of our approach is a set of translation rules that transform a package query to an integer linear program. (3) We introduce an offline data partitioning strategy allowing query evaluation to scale to large data sizes. (4) We introduce SketchRefine, a scalable algorithm for package evaluation, with strong approximation guarantees [(Formula presented.)-factor approximation]. (5) We present a method for parallelizing the Refine phase of SketchRefine. (6) We present an empirical study of the efficiency gains of providing integer solvers with starting solutions. (7) We present extensive experiments over real-world and benchmark data. The results demonstrate that our methods are effective at deriving high-quality package results and achieve runtime performance that is an order of magnitude faster than directly using ILP solvers over large datasets.

KW - Approximation algorithm

KW - Integer linear programming

KW - Package queries

KW - PaQL

KW - SketchRefine

UR - http://www.scopus.com/inward/record.url?scp=85032029590&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85032029590&partnerID=8YFLogxK

U2 - 10.1007/s00778-017-0483-4

DO - 10.1007/s00778-017-0483-4

M3 - Article

AN - SCOPUS:85032029590

SP - 1

EP - 26

JO - VLDB Journal

JF - VLDB Journal

SN - 1066-8888

ER -