Resolution limits of sparse coding in high dimensions

Alyson K. Fletcher, Sundeep Rangan, Vivek K. Goyal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper addresses the problem of sparsity pattern detection for unknown κ-sparse n-dimensional signals observed through m noisy, random linear measurements. Sparsity pattern recovery arises in a number of settings including statistical model selection, pattern detection, and image acquisition. The main results in this paper are necessary and sufficient conditions for asymptotically-reliable sparsity pattern recovery in terms of the dimensions m, n and k as well as the signal-tonoise ratio (SNR) and the minimum-to-average ratio (MAR) of the nonzero entries of the signal. We show that m > 2κ log(n - κ)/(SNR ?MAR) is necessary for any algorithm to succeed, regardless of complexity; this matches a previous sufficient condition for maximum likelihood estimation within a constant factor under certain scalings of κ, SNR and MAR with n. We also show a sufficient condition for a computationally-trivial thresholding algorithm that is larger than the previous expression by only a factor of 4(1+SNR) and larger than the requirement for lasso by only a factor of 4/MAR. This provides insight on the precise value and limitations of convex programming-based algorithms.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
Pages449-456
Number of pages8
StatePublished - 2009
Event22nd Annual Conference on Neural Information Processing Systems, NIPS 2008 - Vancouver, BC, Canada
Duration: Dec 8 2008Dec 11 2008

Other

Other22nd Annual Conference on Neural Information Processing Systems, NIPS 2008
CountryCanada
CityVancouver, BC
Period12/8/0812/11/08

Fingerprint

Recovery
Convex optimization
Maximum likelihood estimation
Image acquisition
Statistical Models

ASJC Scopus subject areas

  • Information Systems

Cite this

Fletcher, A. K., Rangan, S., & Goyal, V. K. (2009). Resolution limits of sparse coding in high dimensions. In Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference (pp. 449-456)

Resolution limits of sparse coding in high dimensions. / Fletcher, Alyson K.; Rangan, Sundeep; Goyal, Vivek K.

Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. 2009. p. 449-456.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fletcher, AK, Rangan, S & Goyal, VK 2009, Resolution limits of sparse coding in high dimensions. in Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. pp. 449-456, 22nd Annual Conference on Neural Information Processing Systems, NIPS 2008, Vancouver, BC, Canada, 12/8/08.
Fletcher AK, Rangan S, Goyal VK. Resolution limits of sparse coding in high dimensions. In Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. 2009. p. 449-456
Fletcher, Alyson K. ; Rangan, Sundeep ; Goyal, Vivek K. / Resolution limits of sparse coding in high dimensions. Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. 2009. pp. 449-456
@inproceedings{d616933f4e7d49f8b73c6b53c909015d,
title = "Resolution limits of sparse coding in high dimensions",
abstract = "This paper addresses the problem of sparsity pattern detection for unknown κ-sparse n-dimensional signals observed through m noisy, random linear measurements. Sparsity pattern recovery arises in a number of settings including statistical model selection, pattern detection, and image acquisition. The main results in this paper are necessary and sufficient conditions for asymptotically-reliable sparsity pattern recovery in terms of the dimensions m, n and k as well as the signal-tonoise ratio (SNR) and the minimum-to-average ratio (MAR) of the nonzero entries of the signal. We show that m > 2κ log(n - κ)/(SNR ?MAR) is necessary for any algorithm to succeed, regardless of complexity; this matches a previous sufficient condition for maximum likelihood estimation within a constant factor under certain scalings of κ, SNR and MAR with n. We also show a sufficient condition for a computationally-trivial thresholding algorithm that is larger than the previous expression by only a factor of 4(1+SNR) and larger than the requirement for lasso by only a factor of 4/MAR. This provides insight on the precise value and limitations of convex programming-based algorithms.",
author = "Fletcher, {Alyson K.} and Sundeep Rangan and Goyal, {Vivek K.}",
year = "2009",
language = "English (US)",
isbn = "9781605609492",
pages = "449--456",
booktitle = "Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference",

}

TY - GEN

T1 - Resolution limits of sparse coding in high dimensions

AU - Fletcher, Alyson K.

AU - Rangan, Sundeep

AU - Goyal, Vivek K.

PY - 2009

Y1 - 2009

N2 - This paper addresses the problem of sparsity pattern detection for unknown κ-sparse n-dimensional signals observed through m noisy, random linear measurements. Sparsity pattern recovery arises in a number of settings including statistical model selection, pattern detection, and image acquisition. The main results in this paper are necessary and sufficient conditions for asymptotically-reliable sparsity pattern recovery in terms of the dimensions m, n and k as well as the signal-tonoise ratio (SNR) and the minimum-to-average ratio (MAR) of the nonzero entries of the signal. We show that m > 2κ log(n - κ)/(SNR ?MAR) is necessary for any algorithm to succeed, regardless of complexity; this matches a previous sufficient condition for maximum likelihood estimation within a constant factor under certain scalings of κ, SNR and MAR with n. We also show a sufficient condition for a computationally-trivial thresholding algorithm that is larger than the previous expression by only a factor of 4(1+SNR) and larger than the requirement for lasso by only a factor of 4/MAR. This provides insight on the precise value and limitations of convex programming-based algorithms.

AB - This paper addresses the problem of sparsity pattern detection for unknown κ-sparse n-dimensional signals observed through m noisy, random linear measurements. Sparsity pattern recovery arises in a number of settings including statistical model selection, pattern detection, and image acquisition. The main results in this paper are necessary and sufficient conditions for asymptotically-reliable sparsity pattern recovery in terms of the dimensions m, n and k as well as the signal-tonoise ratio (SNR) and the minimum-to-average ratio (MAR) of the nonzero entries of the signal. We show that m > 2κ log(n - κ)/(SNR ?MAR) is necessary for any algorithm to succeed, regardless of complexity; this matches a previous sufficient condition for maximum likelihood estimation within a constant factor under certain scalings of κ, SNR and MAR with n. We also show a sufficient condition for a computationally-trivial thresholding algorithm that is larger than the previous expression by only a factor of 4(1+SNR) and larger than the requirement for lasso by only a factor of 4/MAR. This provides insight on the precise value and limitations of convex programming-based algorithms.

UR - http://www.scopus.com/inward/record.url?scp=84858790046&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84858790046&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781605609492

SP - 449

EP - 456

BT - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

ER -