Stochastic coalescence in logarithmic time

Po Shen Loh, Eyal Lubetzky

Research output: Contribution to journalArticle

Abstract

The following distributed coalescence protocol was introduced by Dahlia Malkhi in 2006 motivated by applications in social networking. Initially there are n agents wishing to coalesce into one cluster via a decentralized stochastic process, where each round is as follows: every cluster flips a fair coin to dictate whether it is to issue or accept requests in this round. Issuing a request amounts to contacting a cluster randomly chosen proportionally to its size. A cluster accepting requests is to select an incoming one uniformly (if there are such) and merge with that cluster. Empirical results by Fernandess and Malkhi suggested the protocol concludes in O(log n) rounds with high probability, whereas numerical estimates by Oded Schramm, based on an ingenious analytic approximation, suggested that the coalescence time should be super-logarithmic. Our contribution is a rigorous study of the stochastic coalescence process with two consequences. First, we confirm that the above process indeed requires super-logarithmic time w.h.p., where the inefficient rounds are due to oversized clusters that occasionally develop. Second, we remedy this by showing that a simple modification produces an essentially optimal distributed protocol; if clusters favor their smallest incoming merge request then the process does terminate in O(log n) rounds w.h.p., and simulations show that the new protocol readily outperforms the original one. Our upper bound hinges on a potential function involving the logarithm of the number of clusters and the cluster-susceptibility, carefully chosen to form a supermartingale. The analysis of the lower bound builds upon the novel approach of Schramm which may find additional applications: rather than seeking a single parameter that controls the system behavior, instead one approximates the system by the Laplace transform of the entire cluster-size distribution.

Original languageEnglish (US)
Pages (from-to)492-528
Number of pages37
JournalAnnals of Applied Probability
Volume23
Issue number2
DOIs
StatePublished - Apr 2013

Fingerprint

Coalescence
Logarithmic
Supermartingale
Distributed Protocol
Social Networking
Number of Clusters
Terminate
Flip
Potential Function
Logarithm
Laplace transform
Control Parameter
Susceptibility
Decentralized
Stochastic Processes
Entire
Lower bound
Upper bound
Approximation

Keywords

  • Randomized distributed algorithms
  • Stochastic coalescence processes

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Stochastic coalescence in logarithmic time. / Loh, Po Shen; Lubetzky, Eyal.

In: Annals of Applied Probability, Vol. 23, No. 2, 04.2013, p. 492-528.

Research output: Contribution to journalArticle

Loh, Po Shen ; Lubetzky, Eyal. / Stochastic coalescence in logarithmic time. In: Annals of Applied Probability. 2013 ; Vol. 23, No. 2. pp. 492-528.
@article{9f9d69715e1e4f09b789e74d9226bc99,
title = "Stochastic coalescence in logarithmic time",
abstract = "The following distributed coalescence protocol was introduced by Dahlia Malkhi in 2006 motivated by applications in social networking. Initially there are n agents wishing to coalesce into one cluster via a decentralized stochastic process, where each round is as follows: every cluster flips a fair coin to dictate whether it is to issue or accept requests in this round. Issuing a request amounts to contacting a cluster randomly chosen proportionally to its size. A cluster accepting requests is to select an incoming one uniformly (if there are such) and merge with that cluster. Empirical results by Fernandess and Malkhi suggested the protocol concludes in O(log n) rounds with high probability, whereas numerical estimates by Oded Schramm, based on an ingenious analytic approximation, suggested that the coalescence time should be super-logarithmic. Our contribution is a rigorous study of the stochastic coalescence process with two consequences. First, we confirm that the above process indeed requires super-logarithmic time w.h.p., where the inefficient rounds are due to oversized clusters that occasionally develop. Second, we remedy this by showing that a simple modification produces an essentially optimal distributed protocol; if clusters favor their smallest incoming merge request then the process does terminate in O(log n) rounds w.h.p., and simulations show that the new protocol readily outperforms the original one. Our upper bound hinges on a potential function involving the logarithm of the number of clusters and the cluster-susceptibility, carefully chosen to form a supermartingale. The analysis of the lower bound builds upon the novel approach of Schramm which may find additional applications: rather than seeking a single parameter that controls the system behavior, instead one approximates the system by the Laplace transform of the entire cluster-size distribution.",
keywords = "Randomized distributed algorithms, Stochastic coalescence processes",
author = "Loh, {Po Shen} and Eyal Lubetzky",
year = "2013",
month = "4",
doi = "10.1214/11-AAP832",
language = "English (US)",
volume = "23",
pages = "492--528",
journal = "Annals of Applied Probability",
issn = "1050-5164",
publisher = "Institute of Mathematical Statistics",
number = "2",

}

TY - JOUR

T1 - Stochastic coalescence in logarithmic time

AU - Loh, Po Shen

AU - Lubetzky, Eyal

PY - 2013/4

Y1 - 2013/4

N2 - The following distributed coalescence protocol was introduced by Dahlia Malkhi in 2006 motivated by applications in social networking. Initially there are n agents wishing to coalesce into one cluster via a decentralized stochastic process, where each round is as follows: every cluster flips a fair coin to dictate whether it is to issue or accept requests in this round. Issuing a request amounts to contacting a cluster randomly chosen proportionally to its size. A cluster accepting requests is to select an incoming one uniformly (if there are such) and merge with that cluster. Empirical results by Fernandess and Malkhi suggested the protocol concludes in O(log n) rounds with high probability, whereas numerical estimates by Oded Schramm, based on an ingenious analytic approximation, suggested that the coalescence time should be super-logarithmic. Our contribution is a rigorous study of the stochastic coalescence process with two consequences. First, we confirm that the above process indeed requires super-logarithmic time w.h.p., where the inefficient rounds are due to oversized clusters that occasionally develop. Second, we remedy this by showing that a simple modification produces an essentially optimal distributed protocol; if clusters favor their smallest incoming merge request then the process does terminate in O(log n) rounds w.h.p., and simulations show that the new protocol readily outperforms the original one. Our upper bound hinges on a potential function involving the logarithm of the number of clusters and the cluster-susceptibility, carefully chosen to form a supermartingale. The analysis of the lower bound builds upon the novel approach of Schramm which may find additional applications: rather than seeking a single parameter that controls the system behavior, instead one approximates the system by the Laplace transform of the entire cluster-size distribution.

AB - The following distributed coalescence protocol was introduced by Dahlia Malkhi in 2006 motivated by applications in social networking. Initially there are n agents wishing to coalesce into one cluster via a decentralized stochastic process, where each round is as follows: every cluster flips a fair coin to dictate whether it is to issue or accept requests in this round. Issuing a request amounts to contacting a cluster randomly chosen proportionally to its size. A cluster accepting requests is to select an incoming one uniformly (if there are such) and merge with that cluster. Empirical results by Fernandess and Malkhi suggested the protocol concludes in O(log n) rounds with high probability, whereas numerical estimates by Oded Schramm, based on an ingenious analytic approximation, suggested that the coalescence time should be super-logarithmic. Our contribution is a rigorous study of the stochastic coalescence process with two consequences. First, we confirm that the above process indeed requires super-logarithmic time w.h.p., where the inefficient rounds are due to oversized clusters that occasionally develop. Second, we remedy this by showing that a simple modification produces an essentially optimal distributed protocol; if clusters favor their smallest incoming merge request then the process does terminate in O(log n) rounds w.h.p., and simulations show that the new protocol readily outperforms the original one. Our upper bound hinges on a potential function involving the logarithm of the number of clusters and the cluster-susceptibility, carefully chosen to form a supermartingale. The analysis of the lower bound builds upon the novel approach of Schramm which may find additional applications: rather than seeking a single parameter that controls the system behavior, instead one approximates the system by the Laplace transform of the entire cluster-size distribution.

KW - Randomized distributed algorithms

KW - Stochastic coalescence processes

UR - http://www.scopus.com/inward/record.url?scp=84879771905&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84879771905&partnerID=8YFLogxK

U2 - 10.1214/11-AAP832

DO - 10.1214/11-AAP832

M3 - Article

AN - SCOPUS:84879771905

VL - 23

SP - 492

EP - 528

JO - Annals of Applied Probability

JF - Annals of Applied Probability

SN - 1050-5164

IS - 2

ER -