A catalog of stream processing optimizations

Martin Hirzel, Robert Soulé, Scott Schneider, Bugra Gedik, Robert Grimm

Research output: Contribution to journalArticle

Abstract

Various research communities have independently arrived at stream processing as a programming model for efficient and parallel computing. These communities include digital signal processing, databases, operating systems, and complex event processing. Since each community faces applications with challenging performance requirements, each of them has developed some of the same optimizations, but often with conflicting terminology and unstated assumptions. This article presents a survey of optimizations for stream processing. It is aimed both at users who need to understand and guide the system's optimizer and at implementers who need to make engineering tradeoffs. To consolidate terminology, this article is organized as a catalog, in a style similar to catalogs of design patterns or refactorings. To make assumptions explicit and help understand tradeoffs, each optimization is presented with its safety constraints (when does it preserve correctness?) and a profitability experiment (when does it improve performance?). We hope that this survey will help future streaming system builders to stand on the shoulders of giants from not just their own community.

Original languageEnglish (US)
Article number46
JournalACM Computing Surveys
Volume46
Issue number4
DOIs
StatePublished - 2014

Fingerprint

Stream Processing
Terminology
Optimization
Processing
Trade-offs
Complex Event Processing
Parallel processing systems
Digital signal processing
Refactoring
Design Patterns
Profitability
Parallel Computing
Streaming
Operating Systems
Programming Model
Signal Processing
Correctness
Safety
Engineering
Community

Keywords

  • Optimizations
  • Stream processing

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Hirzel, M., Soulé, R., Schneider, S., Gedik, B., & Grimm, R. (2014). A catalog of stream processing optimizations. ACM Computing Surveys, 46(4), [46]. https://doi.org/10.1145/2528412

A catalog of stream processing optimizations. / Hirzel, Martin; Soulé, Robert; Schneider, Scott; Gedik, Bugra; Grimm, Robert.

In: ACM Computing Surveys, Vol. 46, No. 4, 46, 2014.

Research output: Contribution to journalArticle

Hirzel, M, Soulé, R, Schneider, S, Gedik, B & Grimm, R 2014, 'A catalog of stream processing optimizations', ACM Computing Surveys, vol. 46, no. 4, 46. https://doi.org/10.1145/2528412
Hirzel M, Soulé R, Schneider S, Gedik B, Grimm R. A catalog of stream processing optimizations. ACM Computing Surveys. 2014;46(4). 46. https://doi.org/10.1145/2528412
Hirzel, Martin ; Soulé, Robert ; Schneider, Scott ; Gedik, Bugra ; Grimm, Robert. / A catalog of stream processing optimizations. In: ACM Computing Surveys. 2014 ; Vol. 46, No. 4.
@article{96b11301005a40df85557e8f2390541e,
title = "A catalog of stream processing optimizations",
abstract = "Various research communities have independently arrived at stream processing as a programming model for efficient and parallel computing. These communities include digital signal processing, databases, operating systems, and complex event processing. Since each community faces applications with challenging performance requirements, each of them has developed some of the same optimizations, but often with conflicting terminology and unstated assumptions. This article presents a survey of optimizations for stream processing. It is aimed both at users who need to understand and guide the system's optimizer and at implementers who need to make engineering tradeoffs. To consolidate terminology, this article is organized as a catalog, in a style similar to catalogs of design patterns or refactorings. To make assumptions explicit and help understand tradeoffs, each optimization is presented with its safety constraints (when does it preserve correctness?) and a profitability experiment (when does it improve performance?). We hope that this survey will help future streaming system builders to stand on the shoulders of giants from not just their own community.",
keywords = "Optimizations, Stream processing",
author = "Martin Hirzel and Robert Soul{\'e} and Scott Schneider and Bugra Gedik and Robert Grimm",
year = "2014",
doi = "10.1145/2528412",
language = "English (US)",
volume = "46",
journal = "ACM Computing Surveys",
issn = "0360-0300",
publisher = "Association for Computing Machinery (ACM)",
number = "4",

}

TY - JOUR

T1 - A catalog of stream processing optimizations

AU - Hirzel, Martin

AU - Soulé, Robert

AU - Schneider, Scott

AU - Gedik, Bugra

AU - Grimm, Robert

PY - 2014

Y1 - 2014

N2 - Various research communities have independently arrived at stream processing as a programming model for efficient and parallel computing. These communities include digital signal processing, databases, operating systems, and complex event processing. Since each community faces applications with challenging performance requirements, each of them has developed some of the same optimizations, but often with conflicting terminology and unstated assumptions. This article presents a survey of optimizations for stream processing. It is aimed both at users who need to understand and guide the system's optimizer and at implementers who need to make engineering tradeoffs. To consolidate terminology, this article is organized as a catalog, in a style similar to catalogs of design patterns or refactorings. To make assumptions explicit and help understand tradeoffs, each optimization is presented with its safety constraints (when does it preserve correctness?) and a profitability experiment (when does it improve performance?). We hope that this survey will help future streaming system builders to stand on the shoulders of giants from not just their own community.

AB - Various research communities have independently arrived at stream processing as a programming model for efficient and parallel computing. These communities include digital signal processing, databases, operating systems, and complex event processing. Since each community faces applications with challenging performance requirements, each of them has developed some of the same optimizations, but often with conflicting terminology and unstated assumptions. This article presents a survey of optimizations for stream processing. It is aimed both at users who need to understand and guide the system's optimizer and at implementers who need to make engineering tradeoffs. To consolidate terminology, this article is organized as a catalog, in a style similar to catalogs of design patterns or refactorings. To make assumptions explicit and help understand tradeoffs, each optimization is presented with its safety constraints (when does it preserve correctness?) and a profitability experiment (when does it improve performance?). We hope that this survey will help future streaming system builders to stand on the shoulders of giants from not just their own community.

KW - Optimizations

KW - Stream processing

UR - http://www.scopus.com/inward/record.url?scp=84901236377&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84901236377&partnerID=8YFLogxK

U2 - 10.1145/2528412

DO - 10.1145/2528412

M3 - Article

VL - 46

JO - ACM Computing Surveys

JF - ACM Computing Surveys

SN - 0360-0300

IS - 4

M1 - 46

ER -