Algorithms for distributed functional monitoring

Graham Cormode, Shanmugavelayutham Muthukrishnan, Ke Yi

    Research output: Contribution to journalArticle

    Abstract

    Consider the following problem: We have k players each receiving a stream of items, and communicating with a central coordinator. Let the multiset of items received by player i up until time t be Ai (t). The coordinator's task is to monitor a given function f computed over the union of the inputs Ui Ai (t), continuously at all times t. The goal is to minimize the number of bits communicated between the players and the coordinator. Of interest is the approximate version where the coordinator outputs 1 if f ≥ τ and 0 if f ≤ (1-ε)τ . This defines the (k, f, τ, ε) distributed functional monitoring problem. Functional monitoring problems are fundamental in distributed systems, in particular sensor networks, where we must minimize communication; they also connect to the well-studied streaming model and communication complexity. Yet few formal bounds are known for functional monitoring. We give upper and lower bounds for the (k, f, τ, ε) problem for some of the basic f's. In particular, we study the frequency moments Fp for p = 0, 1, 2. For F0 and F1, we obtain monitoring algorithms with cost almost the same as algorithms that compute the function for a single instance of time. However, for F2 the monitoring problem seems to be much harder than computing the function for a single time instance. We give a carefully constructed multiround algorithm that uses "sketch summaries" at multiple levels of details and solves the (k, F2, τ, ε) problem with communication Õ (k2/ε + k3/23). Our algorithmic techniques are likely to be useful for other functional monitoring problems as well.

    Original languageEnglish (US)
    Article number21
    JournalACM Transactions on Algorithms
    Volume7
    Issue number2
    DOIs
    StatePublished - Mar 1 2011

    Fingerprint

    Monitoring
    Minimise
    Model Complexity
    Communication Complexity
    Multiset
    Streaming
    Sensor Networks
    Distributed Systems
    Upper and Lower Bounds
    Monitor
    Union
    Likely
    Moment
    Computing
    Output
    Costs
    Communication

    Keywords

    • Distributed computing
    • Functional monitoring

    ASJC Scopus subject areas

    • Mathematics (miscellaneous)

    Cite this

    Algorithms for distributed functional monitoring. / Cormode, Graham; Muthukrishnan, Shanmugavelayutham; Yi, Ke.

    In: ACM Transactions on Algorithms, Vol. 7, No. 2, 21, 01.03.2011.

    Research output: Contribution to journalArticle

    Cormode, Graham ; Muthukrishnan, Shanmugavelayutham ; Yi, Ke. / Algorithms for distributed functional monitoring. In: ACM Transactions on Algorithms. 2011 ; Vol. 7, No. 2.
    @article{04d3528cd9974185adbd51d56122ed3a,
    title = "Algorithms for distributed functional monitoring",
    abstract = "Consider the following problem: We have k players each receiving a stream of items, and communicating with a central coordinator. Let the multiset of items received by player i up until time t be Ai (t). The coordinator's task is to monitor a given function f computed over the union of the inputs Ui Ai (t), continuously at all times t. The goal is to minimize the number of bits communicated between the players and the coordinator. Of interest is the approximate version where the coordinator outputs 1 if f ≥ τ and 0 if f ≤ (1-ε)τ . This defines the (k, f, τ, ε) distributed functional monitoring problem. Functional monitoring problems are fundamental in distributed systems, in particular sensor networks, where we must minimize communication; they also connect to the well-studied streaming model and communication complexity. Yet few formal bounds are known for functional monitoring. We give upper and lower bounds for the (k, f, τ, ε) problem for some of the basic f's. In particular, we study the frequency moments Fp for p = 0, 1, 2. For F0 and F1, we obtain monitoring algorithms with cost almost the same as algorithms that compute the function for a single instance of time. However, for F2 the monitoring problem seems to be much harder than computing the function for a single time instance. We give a carefully constructed multiround algorithm that uses {"}sketch summaries{"} at multiple levels of details and solves the (k, F2, τ, ε) problem with communication {\~O} (k2/ε + k3/2/ε3). Our algorithmic techniques are likely to be useful for other functional monitoring problems as well.",
    keywords = "Distributed computing, Functional monitoring",
    author = "Graham Cormode and Shanmugavelayutham Muthukrishnan and Ke Yi",
    year = "2011",
    month = "3",
    day = "1",
    doi = "10.1145/1921659.1921667",
    language = "English (US)",
    volume = "7",
    journal = "ACM Transactions on Algorithms",
    issn = "1549-6325",
    publisher = "Association for Computing Machinery (ACM)",
    number = "2",

    }

    TY - JOUR

    T1 - Algorithms for distributed functional monitoring

    AU - Cormode, Graham

    AU - Muthukrishnan, Shanmugavelayutham

    AU - Yi, Ke

    PY - 2011/3/1

    Y1 - 2011/3/1

    N2 - Consider the following problem: We have k players each receiving a stream of items, and communicating with a central coordinator. Let the multiset of items received by player i up until time t be Ai (t). The coordinator's task is to monitor a given function f computed over the union of the inputs Ui Ai (t), continuously at all times t. The goal is to minimize the number of bits communicated between the players and the coordinator. Of interest is the approximate version where the coordinator outputs 1 if f ≥ τ and 0 if f ≤ (1-ε)τ . This defines the (k, f, τ, ε) distributed functional monitoring problem. Functional monitoring problems are fundamental in distributed systems, in particular sensor networks, where we must minimize communication; they also connect to the well-studied streaming model and communication complexity. Yet few formal bounds are known for functional monitoring. We give upper and lower bounds for the (k, f, τ, ε) problem for some of the basic f's. In particular, we study the frequency moments Fp for p = 0, 1, 2. For F0 and F1, we obtain monitoring algorithms with cost almost the same as algorithms that compute the function for a single instance of time. However, for F2 the monitoring problem seems to be much harder than computing the function for a single time instance. We give a carefully constructed multiround algorithm that uses "sketch summaries" at multiple levels of details and solves the (k, F2, τ, ε) problem with communication Õ (k2/ε + k3/2/ε3). Our algorithmic techniques are likely to be useful for other functional monitoring problems as well.

    AB - Consider the following problem: We have k players each receiving a stream of items, and communicating with a central coordinator. Let the multiset of items received by player i up until time t be Ai (t). The coordinator's task is to monitor a given function f computed over the union of the inputs Ui Ai (t), continuously at all times t. The goal is to minimize the number of bits communicated between the players and the coordinator. Of interest is the approximate version where the coordinator outputs 1 if f ≥ τ and 0 if f ≤ (1-ε)τ . This defines the (k, f, τ, ε) distributed functional monitoring problem. Functional monitoring problems are fundamental in distributed systems, in particular sensor networks, where we must minimize communication; they also connect to the well-studied streaming model and communication complexity. Yet few formal bounds are known for functional monitoring. We give upper and lower bounds for the (k, f, τ, ε) problem for some of the basic f's. In particular, we study the frequency moments Fp for p = 0, 1, 2. For F0 and F1, we obtain monitoring algorithms with cost almost the same as algorithms that compute the function for a single instance of time. However, for F2 the monitoring problem seems to be much harder than computing the function for a single time instance. We give a carefully constructed multiround algorithm that uses "sketch summaries" at multiple levels of details and solves the (k, F2, τ, ε) problem with communication Õ (k2/ε + k3/2/ε3). Our algorithmic techniques are likely to be useful for other functional monitoring problems as well.

    KW - Distributed computing

    KW - Functional monitoring

    UR - http://www.scopus.com/inward/record.url?scp=79953241666&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=79953241666&partnerID=8YFLogxK

    U2 - 10.1145/1921659.1921667

    DO - 10.1145/1921659.1921667

    M3 - Article

    VL - 7

    JO - ACM Transactions on Algorithms

    JF - ACM Transactions on Algorithms

    SN - 1549-6325

    IS - 2

    M1 - 21

    ER -