Detecting failures in distributed systems with the Falcon spy network

Joshua B. Leners, Hao Wu, Wei Lun Hung, Marcos K. Aguilera, Michael Walfish

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A common way for a distributed system to tolerate crashes is to explicitly detect them and then recover from them. Interestingly, detection can take much longer than recovery, as a result of many advances in recovery techniques, making failure detection the dominant factor in these systems' unavailability when a crash occurs. This paper presents the design, implementation, and evaluation of Falcon, a failure detector with several features. First, Falcon's common-case detection time is sub-second, which keeps unavailability low. Second, Falcon is reliable: it never reports a process as down when it is actually up. Third, Falcon sometimes kills to achieve reliable detection but aims to kill the smallest needed component. Falcon achieves these features by coordinating a network of spies, each monitoring a layer of the system. Falcon's main cost is a small amount of platform-specific logic. Falcon is thus the first failure detector that is fast, reliable, and viable. As such, it could change the way that a class of distributed systems is built.

Original languageEnglish (US)
Title of host publicationSOSP'11 - Proceedings of the 23rd ACM Symposium on Operating Systems Principles
Pages279-294
Number of pages16
DOIs
StatePublished - 2011
Event23rd ACM Symposium on Operating Systems Principles, SOSP 2011 - Cascais, Portugal
Duration: Oct 23 2011Oct 26 2011

Other

Other23rd ACM Symposium on Operating Systems Principles, SOSP 2011
CountryPortugal
CityCascais
Period10/23/1110/26/11

Fingerprint

Detectors
Recovery
Monitoring
Costs

Keywords

  • failure detectors
  • high availability
  • layer-specific monitors
  • layer-specific probes
  • reliable detection
  • STONITH

ASJC Scopus subject areas

  • Software

Cite this

Leners, J. B., Wu, H., Hung, W. L., Aguilera, M. K., & Walfish, M. (2011). Detecting failures in distributed systems with the Falcon spy network. In SOSP'11 - Proceedings of the 23rd ACM Symposium on Operating Systems Principles (pp. 279-294) https://doi.org/10.1145/2043556.2043583

Detecting failures in distributed systems with the Falcon spy network. / Leners, Joshua B.; Wu, Hao; Hung, Wei Lun; Aguilera, Marcos K.; Walfish, Michael.

SOSP'11 - Proceedings of the 23rd ACM Symposium on Operating Systems Principles. 2011. p. 279-294.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Leners, JB, Wu, H, Hung, WL, Aguilera, MK & Walfish, M 2011, Detecting failures in distributed systems with the Falcon spy network. in SOSP'11 - Proceedings of the 23rd ACM Symposium on Operating Systems Principles. pp. 279-294, 23rd ACM Symposium on Operating Systems Principles, SOSP 2011, Cascais, Portugal, 10/23/11. https://doi.org/10.1145/2043556.2043583
Leners JB, Wu H, Hung WL, Aguilera MK, Walfish M. Detecting failures in distributed systems with the Falcon spy network. In SOSP'11 - Proceedings of the 23rd ACM Symposium on Operating Systems Principles. 2011. p. 279-294 https://doi.org/10.1145/2043556.2043583
Leners, Joshua B. ; Wu, Hao ; Hung, Wei Lun ; Aguilera, Marcos K. ; Walfish, Michael. / Detecting failures in distributed systems with the Falcon spy network. SOSP'11 - Proceedings of the 23rd ACM Symposium on Operating Systems Principles. 2011. pp. 279-294
@inproceedings{e3521245ce884a31bfbc0e092a320f02,
title = "Detecting failures in distributed systems with the Falcon spy network",
abstract = "A common way for a distributed system to tolerate crashes is to explicitly detect them and then recover from them. Interestingly, detection can take much longer than recovery, as a result of many advances in recovery techniques, making failure detection the dominant factor in these systems' unavailability when a crash occurs. This paper presents the design, implementation, and evaluation of Falcon, a failure detector with several features. First, Falcon's common-case detection time is sub-second, which keeps unavailability low. Second, Falcon is reliable: it never reports a process as down when it is actually up. Third, Falcon sometimes kills to achieve reliable detection but aims to kill the smallest needed component. Falcon achieves these features by coordinating a network of spies, each monitoring a layer of the system. Falcon's main cost is a small amount of platform-specific logic. Falcon is thus the first failure detector that is fast, reliable, and viable. As such, it could change the way that a class of distributed systems is built.",
keywords = "failure detectors, high availability, layer-specific monitors, layer-specific probes, reliable detection, STONITH",
author = "Leners, {Joshua B.} and Hao Wu and Hung, {Wei Lun} and Aguilera, {Marcos K.} and Michael Walfish",
year = "2011",
doi = "10.1145/2043556.2043583",
language = "English (US)",
isbn = "9781450309776",
pages = "279--294",
booktitle = "SOSP'11 - Proceedings of the 23rd ACM Symposium on Operating Systems Principles",

}

TY - GEN

T1 - Detecting failures in distributed systems with the Falcon spy network

AU - Leners, Joshua B.

AU - Wu, Hao

AU - Hung, Wei Lun

AU - Aguilera, Marcos K.

AU - Walfish, Michael

PY - 2011

Y1 - 2011

N2 - A common way for a distributed system to tolerate crashes is to explicitly detect them and then recover from them. Interestingly, detection can take much longer than recovery, as a result of many advances in recovery techniques, making failure detection the dominant factor in these systems' unavailability when a crash occurs. This paper presents the design, implementation, and evaluation of Falcon, a failure detector with several features. First, Falcon's common-case detection time is sub-second, which keeps unavailability low. Second, Falcon is reliable: it never reports a process as down when it is actually up. Third, Falcon sometimes kills to achieve reliable detection but aims to kill the smallest needed component. Falcon achieves these features by coordinating a network of spies, each monitoring a layer of the system. Falcon's main cost is a small amount of platform-specific logic. Falcon is thus the first failure detector that is fast, reliable, and viable. As such, it could change the way that a class of distributed systems is built.

AB - A common way for a distributed system to tolerate crashes is to explicitly detect them and then recover from them. Interestingly, detection can take much longer than recovery, as a result of many advances in recovery techniques, making failure detection the dominant factor in these systems' unavailability when a crash occurs. This paper presents the design, implementation, and evaluation of Falcon, a failure detector with several features. First, Falcon's common-case detection time is sub-second, which keeps unavailability low. Second, Falcon is reliable: it never reports a process as down when it is actually up. Third, Falcon sometimes kills to achieve reliable detection but aims to kill the smallest needed component. Falcon achieves these features by coordinating a network of spies, each monitoring a layer of the system. Falcon's main cost is a small amount of platform-specific logic. Falcon is thus the first failure detector that is fast, reliable, and viable. As such, it could change the way that a class of distributed systems is built.

KW - failure detectors

KW - high availability

KW - layer-specific monitors

KW - layer-specific probes

KW - reliable detection

KW - STONITH

UR - http://www.scopus.com/inward/record.url?scp=82655165279&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=82655165279&partnerID=8YFLogxK

U2 - 10.1145/2043556.2043583

DO - 10.1145/2043556.2043583

M3 - Conference contribution

SN - 9781450309776

SP - 279

EP - 294

BT - SOSP'11 - Proceedings of the 23rd ACM Symposium on Operating Systems Principles

ER -