Nonlinear extraction of independent components of natural images using radial gaussianization

Siwei Lyu, Eero Simoncelli

Research output: Contribution to journalArticle

Abstract

We consider the problem of efficiently encoding a signal by transforming it to a new representation whose components are statistically independent. A widely studied linear solution, known as independent component analysis (ICA), exists for the case when the signal is generated as a linear transformation of independent nongaussian sources. Here, we examine a complementary case, in which the source is nongaussian and elliptically symmetric. In this case, no invertible linear transform suffices to decompose the signal into independent components, but we show that a simple nonlinear transformation, which we call radial gaussianization (RG), is able to remove all dependencies. We then examine this methodology in the context of natural image statistics.We first show that distributions of spatially proximal bandpass filter responses are better described as elliptical than as linearly transformed independent sources. Consistent with this, we demonstrate that the reduction in dependency achieved by applying RG to either nearby pairs or blocks of bandpass filter responses is significantly greater than that achieved by ICA. Finally, we show that the RG transformation may be closely approximated by divisive normalization, which has been used to model the nonlinear response properties of visual neurons.

Original languageEnglish (US)
Pages (from-to)1485-1519
Number of pages35
JournalNeural Computation
Volume21
Issue number6
DOIs
StatePublished - Jun 2009

Fingerprint

Neurons

ASJC Scopus subject areas

  • Cognitive Neuroscience

Cite this

Nonlinear extraction of independent components of natural images using radial gaussianization. / Lyu, Siwei; Simoncelli, Eero.

In: Neural Computation, Vol. 21, No. 6, 06.2009, p. 1485-1519.

Research output: Contribution to journalArticle

@article{8a6fcf16496b4a23b3a4a9d67a83b351,
title = "Nonlinear extraction of independent components of natural images using radial gaussianization",
abstract = "We consider the problem of efficiently encoding a signal by transforming it to a new representation whose components are statistically independent. A widely studied linear solution, known as independent component analysis (ICA), exists for the case when the signal is generated as a linear transformation of independent nongaussian sources. Here, we examine a complementary case, in which the source is nongaussian and elliptically symmetric. In this case, no invertible linear transform suffices to decompose the signal into independent components, but we show that a simple nonlinear transformation, which we call radial gaussianization (RG), is able to remove all dependencies. We then examine this methodology in the context of natural image statistics.We first show that distributions of spatially proximal bandpass filter responses are better described as elliptical than as linearly transformed independent sources. Consistent with this, we demonstrate that the reduction in dependency achieved by applying RG to either nearby pairs or blocks of bandpass filter responses is significantly greater than that achieved by ICA. Finally, we show that the RG transformation may be closely approximated by divisive normalization, which has been used to model the nonlinear response properties of visual neurons.",
author = "Siwei Lyu and Eero Simoncelli",
year = "2009",
month = "6",
doi = "10.1162/neco.2009.04-08-773",
language = "English (US)",
volume = "21",
pages = "1485--1519",
journal = "Neural Computation",
issn = "0899-7667",
publisher = "MIT Press Journals",
number = "6",

}

TY - JOUR

T1 - Nonlinear extraction of independent components of natural images using radial gaussianization

AU - Lyu, Siwei

AU - Simoncelli, Eero

PY - 2009/6

Y1 - 2009/6

N2 - We consider the problem of efficiently encoding a signal by transforming it to a new representation whose components are statistically independent. A widely studied linear solution, known as independent component analysis (ICA), exists for the case when the signal is generated as a linear transformation of independent nongaussian sources. Here, we examine a complementary case, in which the source is nongaussian and elliptically symmetric. In this case, no invertible linear transform suffices to decompose the signal into independent components, but we show that a simple nonlinear transformation, which we call radial gaussianization (RG), is able to remove all dependencies. We then examine this methodology in the context of natural image statistics.We first show that distributions of spatially proximal bandpass filter responses are better described as elliptical than as linearly transformed independent sources. Consistent with this, we demonstrate that the reduction in dependency achieved by applying RG to either nearby pairs or blocks of bandpass filter responses is significantly greater than that achieved by ICA. Finally, we show that the RG transformation may be closely approximated by divisive normalization, which has been used to model the nonlinear response properties of visual neurons.

AB - We consider the problem of efficiently encoding a signal by transforming it to a new representation whose components are statistically independent. A widely studied linear solution, known as independent component analysis (ICA), exists for the case when the signal is generated as a linear transformation of independent nongaussian sources. Here, we examine a complementary case, in which the source is nongaussian and elliptically symmetric. In this case, no invertible linear transform suffices to decompose the signal into independent components, but we show that a simple nonlinear transformation, which we call radial gaussianization (RG), is able to remove all dependencies. We then examine this methodology in the context of natural image statistics.We first show that distributions of spatially proximal bandpass filter responses are better described as elliptical than as linearly transformed independent sources. Consistent with this, we demonstrate that the reduction in dependency achieved by applying RG to either nearby pairs or blocks of bandpass filter responses is significantly greater than that achieved by ICA. Finally, we show that the RG transformation may be closely approximated by divisive normalization, which has been used to model the nonlinear response properties of visual neurons.

UR - http://www.scopus.com/inward/record.url?scp=67651065449&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67651065449&partnerID=8YFLogxK

U2 - 10.1162/neco.2009.04-08-773

DO - 10.1162/neco.2009.04-08-773

M3 - Article

VL - 21

SP - 1485

EP - 1519

JO - Neural Computation

JF - Neural Computation

SN - 0899-7667

IS - 6

ER -