Perceptual straightening of natural videos

Olivier J. Hénaff, Robbe L.T. Goris, Eero P. Simoncelli

Research output: Contribution to journalArticle

Abstract

Many behaviors rely on predictions derived from recent visual input, but the temporal evolution of those inputs is generally complex and difficult to extrapolate. We propose that the visual system transforms these inputs to follow straighter temporal trajectories. To test this ‘temporal straightening’ hypothesis, we develop a methodology for estimating the curvature of an internal trajectory from human perceptual judgments. We use this to test three distinct predictions: natural sequences that are highly curved in the space of pixel intensities should be substantially straighter perceptually; in contrast, artificial sequences that are straight in the intensity domain should be more curved perceptually; finally, naturalistic sequences that are straight in the intensity domain should be relatively less curved. Perceptual data validate all three predictions, as do population models of the early visual system, providing evidence that the visual system specifically straightens natural videos, offering a solution for tasks that rely on prediction.

Original languageEnglish (US)
Pages (from-to)984-991
Number of pages8
JournalNature Neuroscience
Volume22
Issue number6
DOIs
StatePublished - Jun 1 2019

    Fingerprint

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this