Combinatorial algorithms for compressed sensing

Graham Cormode, Shanmugavelayutham Muthukrishnan

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    In sparse approximation theory, the fundamental problem is to reconstruct a signal A ∈ ℝn from linear measurements 〈A,ψi〉 with respect to a dictionary of ψi's. Recently, there is focus on the novel direction of Compressed Sensing [9] where the reconstruction can be done with very few - O(k log n) linear measurements over a modified dictionary if the signal is compressible, that is, its information is concentrated in k coefficients with the original dictionary. In particular, these results [9,4,23] prove that there exists a single O(k log n) × n measurement matrix such that any such signal can be reconstructed from these measurements, with error at most O(1) times the worst case error for the class of such signals. Compressed sensing has generated tremendous excitement both because of the sophisticated underlying Mathematics and because of its potential applications. In this paper, we address outstanding open problems in Compressed Sensing. Our main result is an explicit construction of a non-adaptive measurement matrix and the corresponding reconstruction algorithm so that with a number of measurements polynomial in k, log n, 1/ε, we can reconstruct compressible signals. This is the first known polynomial time explicit construction of any such measurement matrix. In addition, our result improves the error guarantee from O(1) to 1 + ε and improves the reconstruction time from poly(n) to poly(k log n). Our second result is a randomized construction of O(k polylog(n)) measurements that work for each signal with high probability and gives per-instance approximation guarantees rather than over the class of all signals. Previous work on Compressed Sensing does not provide such per-instance approximation guarantees; our result improves the best known number of measurements known from prior work in other areas including Learning Theory [20, 21], Streaming algorithms [11, 12, 6] and Complexity Theory [1] for this case. Our approach is combinatorial. In particular, we use two parallel sets of group tests, one to filter and the other to certify and estimate; the resulting algorithms are quite simple to implement.

    Original languageEnglish (US)
    Title of host publicationStructural Information and Communication Complexity - 13th International Colloquium, SIROCCO 2006, Proceedings
    PublisherSpringer-Verlag
    Pages280-294
    Number of pages15
    ISBN (Print)3540354743, 9783540354741
    DOIs
    StatePublished - Jan 1 2006
    Event13th International Colloquium on Structural Information and Communication Complexity, SIROCCO 2006 - Chester, United Kingdom
    Duration: Jul 2 2006Jul 5 2006

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume4056 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    Conference13th International Colloquium on Structural Information and Communication Complexity, SIROCCO 2006
    CountryUnited Kingdom
    CityChester
    Period7/2/067/5/06

    Fingerprint

    Compressed sensing
    Combinatorial Algorithms
    Compressed Sensing
    Glossaries
    Polynomials
    Sparse Approximation
    Worst Case Error
    Approximation theory
    Learning Theory
    Complexity Theory
    Approximation Theory
    Reconstruction Algorithm
    Approximation
    Streaming
    Open Problems
    Polynomial time
    Filter
    Polynomial

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Computer Science(all)

    Cite this

    Cormode, G., & Muthukrishnan, S. (2006). Combinatorial algorithms for compressed sensing. In Structural Information and Communication Complexity - 13th International Colloquium, SIROCCO 2006, Proceedings (pp. 280-294). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4056 LNCS). Springer-Verlag. https://doi.org/10.1007/11780823_22

    Combinatorial algorithms for compressed sensing. / Cormode, Graham; Muthukrishnan, Shanmugavelayutham.

    Structural Information and Communication Complexity - 13th International Colloquium, SIROCCO 2006, Proceedings. Springer-Verlag, 2006. p. 280-294 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4056 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Cormode, G & Muthukrishnan, S 2006, Combinatorial algorithms for compressed sensing. in Structural Information and Communication Complexity - 13th International Colloquium, SIROCCO 2006, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4056 LNCS, Springer-Verlag, pp. 280-294, 13th International Colloquium on Structural Information and Communication Complexity, SIROCCO 2006, Chester, United Kingdom, 7/2/06. https://doi.org/10.1007/11780823_22
    Cormode G, Muthukrishnan S. Combinatorial algorithms for compressed sensing. In Structural Information and Communication Complexity - 13th International Colloquium, SIROCCO 2006, Proceedings. Springer-Verlag. 2006. p. 280-294. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/11780823_22
    Cormode, Graham ; Muthukrishnan, Shanmugavelayutham. / Combinatorial algorithms for compressed sensing. Structural Information and Communication Complexity - 13th International Colloquium, SIROCCO 2006, Proceedings. Springer-Verlag, 2006. pp. 280-294 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{6cceef1177d649218124df4d30c51631,
    title = "Combinatorial algorithms for compressed sensing",
    abstract = "In sparse approximation theory, the fundamental problem is to reconstruct a signal A ∈ ℝn from linear measurements 〈A,ψi〉 with respect to a dictionary of ψi's. Recently, there is focus on the novel direction of Compressed Sensing [9] where the reconstruction can be done with very few - O(k log n) linear measurements over a modified dictionary if the signal is compressible, that is, its information is concentrated in k coefficients with the original dictionary. In particular, these results [9,4,23] prove that there exists a single O(k log n) × n measurement matrix such that any such signal can be reconstructed from these measurements, with error at most O(1) times the worst case error for the class of such signals. Compressed sensing has generated tremendous excitement both because of the sophisticated underlying Mathematics and because of its potential applications. In this paper, we address outstanding open problems in Compressed Sensing. Our main result is an explicit construction of a non-adaptive measurement matrix and the corresponding reconstruction algorithm so that with a number of measurements polynomial in k, log n, 1/ε, we can reconstruct compressible signals. This is the first known polynomial time explicit construction of any such measurement matrix. In addition, our result improves the error guarantee from O(1) to 1 + ε and improves the reconstruction time from poly(n) to poly(k log n). Our second result is a randomized construction of O(k polylog(n)) measurements that work for each signal with high probability and gives per-instance approximation guarantees rather than over the class of all signals. Previous work on Compressed Sensing does not provide such per-instance approximation guarantees; our result improves the best known number of measurements known from prior work in other areas including Learning Theory [20, 21], Streaming algorithms [11, 12, 6] and Complexity Theory [1] for this case. Our approach is combinatorial. In particular, we use two parallel sets of group tests, one to filter and the other to certify and estimate; the resulting algorithms are quite simple to implement.",
    author = "Graham Cormode and Shanmugavelayutham Muthukrishnan",
    year = "2006",
    month = "1",
    day = "1",
    doi = "10.1007/11780823_22",
    language = "English (US)",
    isbn = "3540354743",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    publisher = "Springer-Verlag",
    pages = "280--294",
    booktitle = "Structural Information and Communication Complexity - 13th International Colloquium, SIROCCO 2006, Proceedings",

    }

    TY - GEN

    T1 - Combinatorial algorithms for compressed sensing

    AU - Cormode, Graham

    AU - Muthukrishnan, Shanmugavelayutham

    PY - 2006/1/1

    Y1 - 2006/1/1

    N2 - In sparse approximation theory, the fundamental problem is to reconstruct a signal A ∈ ℝn from linear measurements 〈A,ψi〉 with respect to a dictionary of ψi's. Recently, there is focus on the novel direction of Compressed Sensing [9] where the reconstruction can be done with very few - O(k log n) linear measurements over a modified dictionary if the signal is compressible, that is, its information is concentrated in k coefficients with the original dictionary. In particular, these results [9,4,23] prove that there exists a single O(k log n) × n measurement matrix such that any such signal can be reconstructed from these measurements, with error at most O(1) times the worst case error for the class of such signals. Compressed sensing has generated tremendous excitement both because of the sophisticated underlying Mathematics and because of its potential applications. In this paper, we address outstanding open problems in Compressed Sensing. Our main result is an explicit construction of a non-adaptive measurement matrix and the corresponding reconstruction algorithm so that with a number of measurements polynomial in k, log n, 1/ε, we can reconstruct compressible signals. This is the first known polynomial time explicit construction of any such measurement matrix. In addition, our result improves the error guarantee from O(1) to 1 + ε and improves the reconstruction time from poly(n) to poly(k log n). Our second result is a randomized construction of O(k polylog(n)) measurements that work for each signal with high probability and gives per-instance approximation guarantees rather than over the class of all signals. Previous work on Compressed Sensing does not provide such per-instance approximation guarantees; our result improves the best known number of measurements known from prior work in other areas including Learning Theory [20, 21], Streaming algorithms [11, 12, 6] and Complexity Theory [1] for this case. Our approach is combinatorial. In particular, we use two parallel sets of group tests, one to filter and the other to certify and estimate; the resulting algorithms are quite simple to implement.

    AB - In sparse approximation theory, the fundamental problem is to reconstruct a signal A ∈ ℝn from linear measurements 〈A,ψi〉 with respect to a dictionary of ψi's. Recently, there is focus on the novel direction of Compressed Sensing [9] where the reconstruction can be done with very few - O(k log n) linear measurements over a modified dictionary if the signal is compressible, that is, its information is concentrated in k coefficients with the original dictionary. In particular, these results [9,4,23] prove that there exists a single O(k log n) × n measurement matrix such that any such signal can be reconstructed from these measurements, with error at most O(1) times the worst case error for the class of such signals. Compressed sensing has generated tremendous excitement both because of the sophisticated underlying Mathematics and because of its potential applications. In this paper, we address outstanding open problems in Compressed Sensing. Our main result is an explicit construction of a non-adaptive measurement matrix and the corresponding reconstruction algorithm so that with a number of measurements polynomial in k, log n, 1/ε, we can reconstruct compressible signals. This is the first known polynomial time explicit construction of any such measurement matrix. In addition, our result improves the error guarantee from O(1) to 1 + ε and improves the reconstruction time from poly(n) to poly(k log n). Our second result is a randomized construction of O(k polylog(n)) measurements that work for each signal with high probability and gives per-instance approximation guarantees rather than over the class of all signals. Previous work on Compressed Sensing does not provide such per-instance approximation guarantees; our result improves the best known number of measurements known from prior work in other areas including Learning Theory [20, 21], Streaming algorithms [11, 12, 6] and Complexity Theory [1] for this case. Our approach is combinatorial. In particular, we use two parallel sets of group tests, one to filter and the other to certify and estimate; the resulting algorithms are quite simple to implement.

    UR - http://www.scopus.com/inward/record.url?scp=33746345633&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=33746345633&partnerID=8YFLogxK

    U2 - 10.1007/11780823_22

    DO - 10.1007/11780823_22

    M3 - Conference contribution

    SN - 3540354743

    SN - 9783540354741

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 280

    EP - 294

    BT - Structural Information and Communication Complexity - 13th International Colloquium, SIROCCO 2006, Proceedings

    PB - Springer-Verlag

    ER -