Virtual lightweight snapshots for consistent analytics in NoSQL stores

Fernando Chirigati, Jerome Simeon, Martin Hirzel, Juliana Freire

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Increasingly, applications that deal with big data need to run analytics concurrently with updates. But bridging the gap between big and fast data is challenging: most of these applications require analytics' results that are fresh and consistent, but without impacting system latency and throughput. We propose virtual lightweight snapshots (VLS), a mechanism that enables consistent analytics without blocking incoming updates in NoSQL stores. VLS requires neither native support for database versioning nor a transaction manager. Besides, it is storage-efficient, keeping additional versions of records only when needed to guarantee consistency, and sharing versions across multiple concurrent snapshots. We describe an implementation of VLS in MongoDB and present a detailed experimental evaluation which shows that it supports consistency for analytics with small impact on query evaluation time, update throughput, and latency.

Original languageEnglish (US)
Title of host publication2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1310-1321
Number of pages12
ISBN (Electronic)9781509020195
DOIs
StatePublished - Jun 22 2016
Event32nd IEEE International Conference on Data Engineering, ICDE 2016 - Helsinki, Finland
Duration: May 16 2016May 20 2016

Other

Other32nd IEEE International Conference on Data Engineering, ICDE 2016
CountryFinland
CityHelsinki
Period5/16/165/20/16

Fingerprint

Throughput
Managers
Big data
Evaluation
Latency

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computational Theory and Mathematics
  • Computer Graphics and Computer-Aided Design
  • Computer Networks and Communications
  • Information Systems
  • Information Systems and Management

Cite this

Chirigati, F., Simeon, J., Hirzel, M., & Freire, J. (2016). Virtual lightweight snapshots for consistent analytics in NoSQL stores. In 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016 (pp. 1310-1321). [7498334] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDE.2016.7498334

Virtual lightweight snapshots for consistent analytics in NoSQL stores. / Chirigati, Fernando; Simeon, Jerome; Hirzel, Martin; Freire, Juliana.

2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 1310-1321 7498334.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Chirigati, F, Simeon, J, Hirzel, M & Freire, J 2016, Virtual lightweight snapshots for consistent analytics in NoSQL stores. in 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016., 7498334, Institute of Electrical and Electronics Engineers Inc., pp. 1310-1321, 32nd IEEE International Conference on Data Engineering, ICDE 2016, Helsinki, Finland, 5/16/16. https://doi.org/10.1109/ICDE.2016.7498334
Chirigati F, Simeon J, Hirzel M, Freire J. Virtual lightweight snapshots for consistent analytics in NoSQL stores. In 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 1310-1321. 7498334 https://doi.org/10.1109/ICDE.2016.7498334
Chirigati, Fernando ; Simeon, Jerome ; Hirzel, Martin ; Freire, Juliana. / Virtual lightweight snapshots for consistent analytics in NoSQL stores. 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 1310-1321
@inproceedings{f879756c554343579c39c461e82ce82f,
title = "Virtual lightweight snapshots for consistent analytics in NoSQL stores",
abstract = "Increasingly, applications that deal with big data need to run analytics concurrently with updates. But bridging the gap between big and fast data is challenging: most of these applications require analytics' results that are fresh and consistent, but without impacting system latency and throughput. We propose virtual lightweight snapshots (VLS), a mechanism that enables consistent analytics without blocking incoming updates in NoSQL stores. VLS requires neither native support for database versioning nor a transaction manager. Besides, it is storage-efficient, keeping additional versions of records only when needed to guarantee consistency, and sharing versions across multiple concurrent snapshots. We describe an implementation of VLS in MongoDB and present a detailed experimental evaluation which shows that it supports consistency for analytics with small impact on query evaluation time, update throughput, and latency.",
author = "Fernando Chirigati and Jerome Simeon and Martin Hirzel and Juliana Freire",
year = "2016",
month = "6",
day = "22",
doi = "10.1109/ICDE.2016.7498334",
language = "English (US)",
pages = "1310--1321",
booktitle = "2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Virtual lightweight snapshots for consistent analytics in NoSQL stores

AU - Chirigati, Fernando

AU - Simeon, Jerome

AU - Hirzel, Martin

AU - Freire, Juliana

PY - 2016/6/22

Y1 - 2016/6/22

N2 - Increasingly, applications that deal with big data need to run analytics concurrently with updates. But bridging the gap between big and fast data is challenging: most of these applications require analytics' results that are fresh and consistent, but without impacting system latency and throughput. We propose virtual lightweight snapshots (VLS), a mechanism that enables consistent analytics without blocking incoming updates in NoSQL stores. VLS requires neither native support for database versioning nor a transaction manager. Besides, it is storage-efficient, keeping additional versions of records only when needed to guarantee consistency, and sharing versions across multiple concurrent snapshots. We describe an implementation of VLS in MongoDB and present a detailed experimental evaluation which shows that it supports consistency for analytics with small impact on query evaluation time, update throughput, and latency.

AB - Increasingly, applications that deal with big data need to run analytics concurrently with updates. But bridging the gap between big and fast data is challenging: most of these applications require analytics' results that are fresh and consistent, but without impacting system latency and throughput. We propose virtual lightweight snapshots (VLS), a mechanism that enables consistent analytics without blocking incoming updates in NoSQL stores. VLS requires neither native support for database versioning nor a transaction manager. Besides, it is storage-efficient, keeping additional versions of records only when needed to guarantee consistency, and sharing versions across multiple concurrent snapshots. We describe an implementation of VLS in MongoDB and present a detailed experimental evaluation which shows that it supports consistency for analytics with small impact on query evaluation time, update throughput, and latency.

UR - http://www.scopus.com/inward/record.url?scp=84980392030&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84980392030&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2016.7498334

DO - 10.1109/ICDE.2016.7498334

M3 - Conference contribution

SP - 1310

EP - 1321

BT - 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -