Filtering algorithms and implementation for very fast publish/subscribe systems

Françoise Fahret, H. Arno Jacobsen, François Llirbat, João Pereira, Kenneth A. Ross, Dennis Shasha

Research output: Contribution to journalArticle

Abstract

Publish/Subscribe is the paradigm in which users express long-term interests ("subscriptions") and some agent "publishes" events (e.g., offers). The job of Publish/Subscribe software is to send events to the owners of subscriptions satisfied by those events. For example, a user subscription may consist of an interest in an airplane of a certain type, not to exceed a certain price. A published event may consist of an offer of an airplane with certain properties including price. Each subscription consists of a conjunction of (attribute, comparison operator, value) predicates. A subscription closely resembles a trigger in that it is a long-lived conditional query associated with an action (usually, informing the subscriber). However, it is less general than a trigger so novel data structures and implementations may enable the creation of more scalable, high performance publish/subscribe systems. This paper describes an attempt at the construction of such algorithms and its implementation. Using a combination of data structures, application-specific caching policies, and application-specific query processing our system can handle 600 events per second for a typical workload containing 6 million subscriptions.

Original languageEnglish (US)
Pages (from-to)115-126
Number of pages12
JournalSIGMOD Record (ACM Special Interest Group on Management of Data)
Volume30
Issue number2
StatePublished - Jun 2001

Fingerprint

Data structures
Aircraft
Query processing

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Information Systems
  • Software

Cite this

Filtering algorithms and implementation for very fast publish/subscribe systems. / Fahret, Françoise; Jacobsen, H. Arno; Llirbat, François; Pereira, João; Ross, Kenneth A.; Shasha, Dennis.

In: SIGMOD Record (ACM Special Interest Group on Management of Data), Vol. 30, No. 2, 06.2001, p. 115-126.

Research output: Contribution to journalArticle

Fahret, Françoise ; Jacobsen, H. Arno ; Llirbat, François ; Pereira, João ; Ross, Kenneth A. ; Shasha, Dennis. / Filtering algorithms and implementation for very fast publish/subscribe systems. In: SIGMOD Record (ACM Special Interest Group on Management of Data). 2001 ; Vol. 30, No. 2. pp. 115-126.
@article{746859cc306d46d18a5142abff47c35a,
title = "Filtering algorithms and implementation for very fast publish/subscribe systems",
abstract = "Publish/Subscribe is the paradigm in which users express long-term interests ({"}subscriptions{"}) and some agent {"}publishes{"} events (e.g., offers). The job of Publish/Subscribe software is to send events to the owners of subscriptions satisfied by those events. For example, a user subscription may consist of an interest in an airplane of a certain type, not to exceed a certain price. A published event may consist of an offer of an airplane with certain properties including price. Each subscription consists of a conjunction of (attribute, comparison operator, value) predicates. A subscription closely resembles a trigger in that it is a long-lived conditional query associated with an action (usually, informing the subscriber). However, it is less general than a trigger so novel data structures and implementations may enable the creation of more scalable, high performance publish/subscribe systems. This paper describes an attempt at the construction of such algorithms and its implementation. Using a combination of data structures, application-specific caching policies, and application-specific query processing our system can handle 600 events per second for a typical workload containing 6 million subscriptions.",
author = "Fran{\cc}oise Fahret and Jacobsen, {H. Arno} and Fran{\cc}ois Llirbat and Jo{\~a}o Pereira and Ross, {Kenneth A.} and Dennis Shasha",
year = "2001",
month = "6",
language = "English (US)",
volume = "30",
pages = "115--126",
journal = "SIGMOD Record",
issn = "0163-5808",
publisher = "Association for Computing Machinery (ACM)",
number = "2",

}

TY - JOUR

T1 - Filtering algorithms and implementation for very fast publish/subscribe systems

AU - Fahret, Françoise

AU - Jacobsen, H. Arno

AU - Llirbat, François

AU - Pereira, João

AU - Ross, Kenneth A.

AU - Shasha, Dennis

PY - 2001/6

Y1 - 2001/6

N2 - Publish/Subscribe is the paradigm in which users express long-term interests ("subscriptions") and some agent "publishes" events (e.g., offers). The job of Publish/Subscribe software is to send events to the owners of subscriptions satisfied by those events. For example, a user subscription may consist of an interest in an airplane of a certain type, not to exceed a certain price. A published event may consist of an offer of an airplane with certain properties including price. Each subscription consists of a conjunction of (attribute, comparison operator, value) predicates. A subscription closely resembles a trigger in that it is a long-lived conditional query associated with an action (usually, informing the subscriber). However, it is less general than a trigger so novel data structures and implementations may enable the creation of more scalable, high performance publish/subscribe systems. This paper describes an attempt at the construction of such algorithms and its implementation. Using a combination of data structures, application-specific caching policies, and application-specific query processing our system can handle 600 events per second for a typical workload containing 6 million subscriptions.

AB - Publish/Subscribe is the paradigm in which users express long-term interests ("subscriptions") and some agent "publishes" events (e.g., offers). The job of Publish/Subscribe software is to send events to the owners of subscriptions satisfied by those events. For example, a user subscription may consist of an interest in an airplane of a certain type, not to exceed a certain price. A published event may consist of an offer of an airplane with certain properties including price. Each subscription consists of a conjunction of (attribute, comparison operator, value) predicates. A subscription closely resembles a trigger in that it is a long-lived conditional query associated with an action (usually, informing the subscriber). However, it is less general than a trigger so novel data structures and implementations may enable the creation of more scalable, high performance publish/subscribe systems. This paper describes an attempt at the construction of such algorithms and its implementation. Using a combination of data structures, application-specific caching policies, and application-specific query processing our system can handle 600 events per second for a typical workload containing 6 million subscriptions.

UR - http://www.scopus.com/inward/record.url?scp=0038969993&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0038969993&partnerID=8YFLogxK

M3 - Article

VL - 30

SP - 115

EP - 126

JO - SIGMOD Record

JF - SIGMOD Record

SN - 0163-5808

IS - 2

ER -