Efficient elastic burst detection in data streams

Yunyue Zhu, Dennis Shasha

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Burst detection is the activity of finding abnormal aggregates in data streams. Such aggregates are based on sliding windows over data streams. In some applications, we want to monitor many sliding window sizes simultaneously and to report those windows with aggregates significantly different from other periods. We will present a general data structure for detecting interesting aggregates over such elastic windows in near linear time. We present applications of the algorithm for detecting Gamma Ray Bursts in large-scale astrophysical data. Detection of periods with high volumes of trading activities and high stock price volatility is also demonstrated using real time Trade and Quote (TAQ) data from the New York Stock Exchange (NYSE). Our algorithm beats the direct computation approach by several orders of magnitude.

Original languageEnglish (US)
Title of host publicationProceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03
Pages336-345
Number of pages10
DOIs
StatePublished - 2003
Event9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03 - Washington, DC, United States
Duration: Aug 24 2003Aug 27 2003

Other

Other9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03
CountryUnited States
CityWashington, DC
Period8/24/038/27/03

Fingerprint

Gamma rays
Data structures

Keywords

  • Data stream
  • Elastic burst

ASJC Scopus subject areas

  • Software
  • Information Systems

Cite this

Zhu, Y., & Shasha, D. (2003). Efficient elastic burst detection in data streams. In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03 (pp. 336-345) https://doi.org/10.1145/956750.956789

Efficient elastic burst detection in data streams. / Zhu, Yunyue; Shasha, Dennis.

Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03. 2003. p. 336-345.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhu, Y & Shasha, D 2003, Efficient elastic burst detection in data streams. in Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03. pp. 336-345, 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03, Washington, DC, United States, 8/24/03. https://doi.org/10.1145/956750.956789
Zhu Y, Shasha D. Efficient elastic burst detection in data streams. In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03. 2003. p. 336-345 https://doi.org/10.1145/956750.956789
Zhu, Yunyue ; Shasha, Dennis. / Efficient elastic burst detection in data streams. Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03. 2003. pp. 336-345
@inproceedings{a2f66917873b45b5bfb0b9823d1179cf,
title = "Efficient elastic burst detection in data streams",
abstract = "Burst detection is the activity of finding abnormal aggregates in data streams. Such aggregates are based on sliding windows over data streams. In some applications, we want to monitor many sliding window sizes simultaneously and to report those windows with aggregates significantly different from other periods. We will present a general data structure for detecting interesting aggregates over such elastic windows in near linear time. We present applications of the algorithm for detecting Gamma Ray Bursts in large-scale astrophysical data. Detection of periods with high volumes of trading activities and high stock price volatility is also demonstrated using real time Trade and Quote (TAQ) data from the New York Stock Exchange (NYSE). Our algorithm beats the direct computation approach by several orders of magnitude.",
keywords = "Data stream, Elastic burst",
author = "Yunyue Zhu and Dennis Shasha",
year = "2003",
doi = "10.1145/956750.956789",
language = "English (US)",
pages = "336--345",
booktitle = "Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03",

}

TY - GEN

T1 - Efficient elastic burst detection in data streams

AU - Zhu, Yunyue

AU - Shasha, Dennis

PY - 2003

Y1 - 2003

N2 - Burst detection is the activity of finding abnormal aggregates in data streams. Such aggregates are based on sliding windows over data streams. In some applications, we want to monitor many sliding window sizes simultaneously and to report those windows with aggregates significantly different from other periods. We will present a general data structure for detecting interesting aggregates over such elastic windows in near linear time. We present applications of the algorithm for detecting Gamma Ray Bursts in large-scale astrophysical data. Detection of periods with high volumes of trading activities and high stock price volatility is also demonstrated using real time Trade and Quote (TAQ) data from the New York Stock Exchange (NYSE). Our algorithm beats the direct computation approach by several orders of magnitude.

AB - Burst detection is the activity of finding abnormal aggregates in data streams. Such aggregates are based on sliding windows over data streams. In some applications, we want to monitor many sliding window sizes simultaneously and to report those windows with aggregates significantly different from other periods. We will present a general data structure for detecting interesting aggregates over such elastic windows in near linear time. We present applications of the algorithm for detecting Gamma Ray Bursts in large-scale astrophysical data. Detection of periods with high volumes of trading activities and high stock price volatility is also demonstrated using real time Trade and Quote (TAQ) data from the New York Stock Exchange (NYSE). Our algorithm beats the direct computation approach by several orders of magnitude.

KW - Data stream

KW - Elastic burst

UR - http://www.scopus.com/inward/record.url?scp=77952383186&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77952383186&partnerID=8YFLogxK

U2 - 10.1145/956750.956789

DO - 10.1145/956750.956789

M3 - Conference contribution

AN - SCOPUS:77952383186

SP - 336

EP - 345

BT - Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03

ER -