BlinkDB: Queries with bounded errors and bounded response times on very large data

Sameer Agarwal, Barzan Mozafari, Aurojit Panda, Henry Milner, Samuel Madden, Ion Stoica

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we present BlinkDB, a massively parallel, approximate query engine for running interactive SQL queries on large volumes of data. BlinkDB allows users to trade-off query accuracy for response time, enabling interactive queries over massive data by running queries on data samples and presenting results annotated with meaningful error bars. To achieve this, BlinkDB uses two key ideas: (1) an adaptive optimization framework that builds and maintains a set of multi-dimensional stratified samples from original data over time, and (2) a dynamic sample selection strategy that selects an appropriately sized sample based on a query's accuracy or response time requirements. We evaluate BlinkDB against the well-known TPC-H benchmarks and a real-world analytic workload derived from Conviva Inc., a company that manages video distribution over the Internet. Our experiments on a 100 node cluster show that BlinkDB can answer queries on up to 17 TBs of data in less than 2 seconds (over 200 x faster than Hive), within an error of 2-10%.

Original languageEnglish (US)
Title of host publicationProceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013
Pages29-42
Number of pages14
DOIs
StatePublished - May 17 2013
Event8th ACM European Conference on Computer Systems, EuroSys 2013 - Prague, Czech Republic
Duration: Apr 15 2013Apr 17 2013

Other

Other8th ACM European Conference on Computer Systems, EuroSys 2013
CountryCzech Republic
CityPrague
Period4/15/134/17/13

Fingerprint

Internet
Engines
Industry
Experiments

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering

Cite this

Agarwal, S., Mozafari, B., Panda, A., Milner, H., Madden, S., & Stoica, I. (2013). BlinkDB: Queries with bounded errors and bounded response times on very large data. In Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013 (pp. 29-42) https://doi.org/10.1145/2465351.2465355

BlinkDB : Queries with bounded errors and bounded response times on very large data. / Agarwal, Sameer; Mozafari, Barzan; Panda, Aurojit; Milner, Henry; Madden, Samuel; Stoica, Ion.

Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013. 2013. p. 29-42.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Agarwal, S, Mozafari, B, Panda, A, Milner, H, Madden, S & Stoica, I 2013, BlinkDB: Queries with bounded errors and bounded response times on very large data. in Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013. pp. 29-42, 8th ACM European Conference on Computer Systems, EuroSys 2013, Prague, Czech Republic, 4/15/13. https://doi.org/10.1145/2465351.2465355
Agarwal S, Mozafari B, Panda A, Milner H, Madden S, Stoica I. BlinkDB: Queries with bounded errors and bounded response times on very large data. In Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013. 2013. p. 29-42 https://doi.org/10.1145/2465351.2465355
Agarwal, Sameer ; Mozafari, Barzan ; Panda, Aurojit ; Milner, Henry ; Madden, Samuel ; Stoica, Ion. / BlinkDB : Queries with bounded errors and bounded response times on very large data. Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013. 2013. pp. 29-42
@inproceedings{81f9d1a4d4794d9491efc7de28d1ff49,
title = "BlinkDB: Queries with bounded errors and bounded response times on very large data",
abstract = "In this paper, we present BlinkDB, a massively parallel, approximate query engine for running interactive SQL queries on large volumes of data. BlinkDB allows users to trade-off query accuracy for response time, enabling interactive queries over massive data by running queries on data samples and presenting results annotated with meaningful error bars. To achieve this, BlinkDB uses two key ideas: (1) an adaptive optimization framework that builds and maintains a set of multi-dimensional stratified samples from original data over time, and (2) a dynamic sample selection strategy that selects an appropriately sized sample based on a query's accuracy or response time requirements. We evaluate BlinkDB against the well-known TPC-H benchmarks and a real-world analytic workload derived from Conviva Inc., a company that manages video distribution over the Internet. Our experiments on a 100 node cluster show that BlinkDB can answer queries on up to 17 TBs of data in less than 2 seconds (over 200 x faster than Hive), within an error of 2-10{\%}.",
author = "Sameer Agarwal and Barzan Mozafari and Aurojit Panda and Henry Milner and Samuel Madden and Ion Stoica",
year = "2013",
month = "5",
day = "17",
doi = "10.1145/2465351.2465355",
language = "English (US)",
isbn = "9781450319942",
pages = "29--42",
booktitle = "Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013",

}

TY - GEN

T1 - BlinkDB

T2 - Queries with bounded errors and bounded response times on very large data

AU - Agarwal, Sameer

AU - Mozafari, Barzan

AU - Panda, Aurojit

AU - Milner, Henry

AU - Madden, Samuel

AU - Stoica, Ion

PY - 2013/5/17

Y1 - 2013/5/17

N2 - In this paper, we present BlinkDB, a massively parallel, approximate query engine for running interactive SQL queries on large volumes of data. BlinkDB allows users to trade-off query accuracy for response time, enabling interactive queries over massive data by running queries on data samples and presenting results annotated with meaningful error bars. To achieve this, BlinkDB uses two key ideas: (1) an adaptive optimization framework that builds and maintains a set of multi-dimensional stratified samples from original data over time, and (2) a dynamic sample selection strategy that selects an appropriately sized sample based on a query's accuracy or response time requirements. We evaluate BlinkDB against the well-known TPC-H benchmarks and a real-world analytic workload derived from Conviva Inc., a company that manages video distribution over the Internet. Our experiments on a 100 node cluster show that BlinkDB can answer queries on up to 17 TBs of data in less than 2 seconds (over 200 x faster than Hive), within an error of 2-10%.

AB - In this paper, we present BlinkDB, a massively parallel, approximate query engine for running interactive SQL queries on large volumes of data. BlinkDB allows users to trade-off query accuracy for response time, enabling interactive queries over massive data by running queries on data samples and presenting results annotated with meaningful error bars. To achieve this, BlinkDB uses two key ideas: (1) an adaptive optimization framework that builds and maintains a set of multi-dimensional stratified samples from original data over time, and (2) a dynamic sample selection strategy that selects an appropriately sized sample based on a query's accuracy or response time requirements. We evaluate BlinkDB against the well-known TPC-H benchmarks and a real-world analytic workload derived from Conviva Inc., a company that manages video distribution over the Internet. Our experiments on a 100 node cluster show that BlinkDB can answer queries on up to 17 TBs of data in less than 2 seconds (over 200 x faster than Hive), within an error of 2-10%.

UR - http://www.scopus.com/inward/record.url?scp=84877703682&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84877703682&partnerID=8YFLogxK

U2 - 10.1145/2465351.2465355

DO - 10.1145/2465351.2465355

M3 - Conference contribution

AN - SCOPUS:84877703682

SN - 9781450319942

SP - 29

EP - 42

BT - Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013

ER -