Perceptual straightening of natural videos

Olivier J. Hénaff, Robbe L.T. Goris, Eero Simoncelli

Research output: Contribution to journalArticle

Abstract

Many behaviors rely on predictions derived from recent visual input, but the temporal evolution of those inputs is generally complex and difficult to extrapolate. We propose that the visual system transforms these inputs to follow straighter temporal trajectories. To test this ‘temporal straightening’ hypothesis, we develop a methodology for estimating the curvature of an internal trajectory from human perceptual judgments. We use this to test three distinct predictions: natural sequences that are highly curved in the space of pixel intensities should be substantially straighter perceptually; in contrast, artificial sequences that are straight in the intensity domain should be more curved perceptually; finally, naturalistic sequences that are straight in the intensity domain should be relatively less curved. Perceptual data validate all three predictions, as do population models of the early visual system, providing evidence that the visual system specifically straightens natural videos, offering a solution for tasks that rely on prediction.

Original languageEnglish (US)
JournalNature Neuroscience
DOIs
StatePublished - Jan 1 2019

Fingerprint

Population

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Perceptual straightening of natural videos. / Hénaff, Olivier J.; Goris, Robbe L.T.; Simoncelli, Eero.

In: Nature Neuroscience, 01.01.2019.

Research output: Contribution to journalArticle

Hénaff, Olivier J. ; Goris, Robbe L.T. ; Simoncelli, Eero. / Perceptual straightening of natural videos. In: Nature Neuroscience. 2019.
@article{883023c9b09840019468a7084eeb1470,
title = "Perceptual straightening of natural videos",
abstract = "Many behaviors rely on predictions derived from recent visual input, but the temporal evolution of those inputs is generally complex and difficult to extrapolate. We propose that the visual system transforms these inputs to follow straighter temporal trajectories. To test this ‘temporal straightening’ hypothesis, we develop a methodology for estimating the curvature of an internal trajectory from human perceptual judgments. We use this to test three distinct predictions: natural sequences that are highly curved in the space of pixel intensities should be substantially straighter perceptually; in contrast, artificial sequences that are straight in the intensity domain should be more curved perceptually; finally, naturalistic sequences that are straight in the intensity domain should be relatively less curved. Perceptual data validate all three predictions, as do population models of the early visual system, providing evidence that the visual system specifically straightens natural videos, offering a solution for tasks that rely on prediction.",
author = "H{\'e}naff, {Olivier J.} and Goris, {Robbe L.T.} and Eero Simoncelli",
year = "2019",
month = "1",
day = "1",
doi = "10.1038/s41593-019-0377-4",
language = "English (US)",
journal = "Nature Neuroscience",
issn = "1097-6256",
publisher = "Nature Publishing Group",

}

TY - JOUR

T1 - Perceptual straightening of natural videos

AU - Hénaff, Olivier J.

AU - Goris, Robbe L.T.

AU - Simoncelli, Eero

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Many behaviors rely on predictions derived from recent visual input, but the temporal evolution of those inputs is generally complex and difficult to extrapolate. We propose that the visual system transforms these inputs to follow straighter temporal trajectories. To test this ‘temporal straightening’ hypothesis, we develop a methodology for estimating the curvature of an internal trajectory from human perceptual judgments. We use this to test three distinct predictions: natural sequences that are highly curved in the space of pixel intensities should be substantially straighter perceptually; in contrast, artificial sequences that are straight in the intensity domain should be more curved perceptually; finally, naturalistic sequences that are straight in the intensity domain should be relatively less curved. Perceptual data validate all three predictions, as do population models of the early visual system, providing evidence that the visual system specifically straightens natural videos, offering a solution for tasks that rely on prediction.

AB - Many behaviors rely on predictions derived from recent visual input, but the temporal evolution of those inputs is generally complex and difficult to extrapolate. We propose that the visual system transforms these inputs to follow straighter temporal trajectories. To test this ‘temporal straightening’ hypothesis, we develop a methodology for estimating the curvature of an internal trajectory from human perceptual judgments. We use this to test three distinct predictions: natural sequences that are highly curved in the space of pixel intensities should be substantially straighter perceptually; in contrast, artificial sequences that are straight in the intensity domain should be more curved perceptually; finally, naturalistic sequences that are straight in the intensity domain should be relatively less curved. Perceptual data validate all three predictions, as do population models of the early visual system, providing evidence that the visual system specifically straightens natural videos, offering a solution for tasks that rely on prediction.

UR - http://www.scopus.com/inward/record.url?scp=85065159738&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065159738&partnerID=8YFLogxK

U2 - 10.1038/s41593-019-0377-4

DO - 10.1038/s41593-019-0377-4

M3 - Article

C2 - 31036946

AN - SCOPUS:85065159738

JO - Nature Neuroscience

JF - Nature Neuroscience

SN - 1097-6256

ER -