A universal calculus for stream processing languages

Robert Soulé, Martin Hirzel, Robert Grimm, Buǧra Gedik, Henrique Andrade, Vibhore Kumar, Kun Lung Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Stream processing applications such as algorithmic trading, MPEG processing, and web content analysis are ubiquitous and essential to business and entertainment. Language designers have developed numerous domain-specific languages that are both tailored to the needs of their applications, and optimized for performance on their particular target platforms. Unfortunately, the goals of generality and performance are frequently at odds, and prior work on the formal semantics of stream processing languages does not capture the details necessary for reasoning about implementations. This paper presents Brooklet, a core calculus for stream processing that allows us to reason about how to map languages to platforms and how to optimize stream programs. We translate from three representative languages, CQL, StreamIt, and Sawzall, to Brooklet, and show that the translations are correct. We formalize three popular and vital optimizations, data-parallel computation, operator fusion, and operator re-ordering, and show under which conditions they are correct. Language designers can use Brooklet to specify exactly how new features or languages behave. Language implementors can use Brooklet to show exactly under which circumstances new optimizations are correct. In ongoing work, we are developing an intermediate language for streaming that is based on Brooklet. We are implementing our intermediate language on System S, IBM's high-performance streaming middleware.

Original languageEnglish (US)
Title of host publicationProgramming Languages and Systems - 19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010, Proceedings
Pages507-528
Number of pages22
Volume6012 LNCS
DOIs
StatePublished - 2010
Event19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010 - Paphos, Cyprus
Duration: Mar 20 2010Mar 28 2010

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume6012 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010
CountryCyprus
CityPaphos
Period3/20/103/28/10

Fingerprint

Stream Processing
Calculus
Processing
Mathematical operators
Middleware
Streaming
Fusion reactions
Semantics
Language
S-system
Odds
Content Analysis
Domain-specific Languages
Formal Semantics
Optimization
Reordering
Parallel Computation
Operator
Industry
Fusion

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Soulé, R., Hirzel, M., Grimm, R., Gedik, B., Andrade, H., Kumar, V., & Wu, K. L. (2010). A universal calculus for stream processing languages. In Programming Languages and Systems - 19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010, Proceedings (Vol. 6012 LNCS, pp. 507-528). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6012 LNCS). https://doi.org/10.1007/978-3-642-11957-6_27

A universal calculus for stream processing languages. / Soulé, Robert; Hirzel, Martin; Grimm, Robert; Gedik, Buǧra; Andrade, Henrique; Kumar, Vibhore; Wu, Kun Lung.

Programming Languages and Systems - 19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010, Proceedings. Vol. 6012 LNCS 2010. p. 507-528 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6012 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Soulé, R, Hirzel, M, Grimm, R, Gedik, B, Andrade, H, Kumar, V & Wu, KL 2010, A universal calculus for stream processing languages. in Programming Languages and Systems - 19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010, Proceedings. vol. 6012 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6012 LNCS, pp. 507-528, 19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010, Paphos, Cyprus, 3/20/10. https://doi.org/10.1007/978-3-642-11957-6_27
Soulé R, Hirzel M, Grimm R, Gedik B, Andrade H, Kumar V et al. A universal calculus for stream processing languages. In Programming Languages and Systems - 19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010, Proceedings. Vol. 6012 LNCS. 2010. p. 507-528. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-11957-6_27
Soulé, Robert ; Hirzel, Martin ; Grimm, Robert ; Gedik, Buǧra ; Andrade, Henrique ; Kumar, Vibhore ; Wu, Kun Lung. / A universal calculus for stream processing languages. Programming Languages and Systems - 19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010, Proceedings. Vol. 6012 LNCS 2010. pp. 507-528 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{8a7b1ae6b98b4251847580268862246f,
title = "A universal calculus for stream processing languages",
abstract = "Stream processing applications such as algorithmic trading, MPEG processing, and web content analysis are ubiquitous and essential to business and entertainment. Language designers have developed numerous domain-specific languages that are both tailored to the needs of their applications, and optimized for performance on their particular target platforms. Unfortunately, the goals of generality and performance are frequently at odds, and prior work on the formal semantics of stream processing languages does not capture the details necessary for reasoning about implementations. This paper presents Brooklet, a core calculus for stream processing that allows us to reason about how to map languages to platforms and how to optimize stream programs. We translate from three representative languages, CQL, StreamIt, and Sawzall, to Brooklet, and show that the translations are correct. We formalize three popular and vital optimizations, data-parallel computation, operator fusion, and operator re-ordering, and show under which conditions they are correct. Language designers can use Brooklet to specify exactly how new features or languages behave. Language implementors can use Brooklet to show exactly under which circumstances new optimizations are correct. In ongoing work, we are developing an intermediate language for streaming that is based on Brooklet. We are implementing our intermediate language on System S, IBM's high-performance streaming middleware.",
author = "Robert Soul{\'e} and Martin Hirzel and Robert Grimm and Buǧra Gedik and Henrique Andrade and Vibhore Kumar and Wu, {Kun Lung}",
year = "2010",
doi = "10.1007/978-3-642-11957-6_27",
language = "English (US)",
isbn = "3642119565",
volume = "6012 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "507--528",
booktitle = "Programming Languages and Systems - 19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010, Proceedings",

}

TY - GEN

T1 - A universal calculus for stream processing languages

AU - Soulé, Robert

AU - Hirzel, Martin

AU - Grimm, Robert

AU - Gedik, Buǧra

AU - Andrade, Henrique

AU - Kumar, Vibhore

AU - Wu, Kun Lung

PY - 2010

Y1 - 2010

N2 - Stream processing applications such as algorithmic trading, MPEG processing, and web content analysis are ubiquitous and essential to business and entertainment. Language designers have developed numerous domain-specific languages that are both tailored to the needs of their applications, and optimized for performance on their particular target platforms. Unfortunately, the goals of generality and performance are frequently at odds, and prior work on the formal semantics of stream processing languages does not capture the details necessary for reasoning about implementations. This paper presents Brooklet, a core calculus for stream processing that allows us to reason about how to map languages to platforms and how to optimize stream programs. We translate from three representative languages, CQL, StreamIt, and Sawzall, to Brooklet, and show that the translations are correct. We formalize three popular and vital optimizations, data-parallel computation, operator fusion, and operator re-ordering, and show under which conditions they are correct. Language designers can use Brooklet to specify exactly how new features or languages behave. Language implementors can use Brooklet to show exactly under which circumstances new optimizations are correct. In ongoing work, we are developing an intermediate language for streaming that is based on Brooklet. We are implementing our intermediate language on System S, IBM's high-performance streaming middleware.

AB - Stream processing applications such as algorithmic trading, MPEG processing, and web content analysis are ubiquitous and essential to business and entertainment. Language designers have developed numerous domain-specific languages that are both tailored to the needs of their applications, and optimized for performance on their particular target platforms. Unfortunately, the goals of generality and performance are frequently at odds, and prior work on the formal semantics of stream processing languages does not capture the details necessary for reasoning about implementations. This paper presents Brooklet, a core calculus for stream processing that allows us to reason about how to map languages to platforms and how to optimize stream programs. We translate from three representative languages, CQL, StreamIt, and Sawzall, to Brooklet, and show that the translations are correct. We formalize three popular and vital optimizations, data-parallel computation, operator fusion, and operator re-ordering, and show under which conditions they are correct. Language designers can use Brooklet to specify exactly how new features or languages behave. Language implementors can use Brooklet to show exactly under which circumstances new optimizations are correct. In ongoing work, we are developing an intermediate language for streaming that is based on Brooklet. We are implementing our intermediate language on System S, IBM's high-performance streaming middleware.

UR - http://www.scopus.com/inward/record.url?scp=77951270754&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77951270754&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-11957-6_27

DO - 10.1007/978-3-642-11957-6_27

M3 - Conference contribution

SN - 3642119565

SN - 9783642119569

VL - 6012 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 507

EP - 528

BT - Programming Languages and Systems - 19th European Symposium on Programming, ESOP 2010, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2010, Proceedings

ER -