Segmentation in structure from motion

Modeling and psychophysics

Corrado Caudek, Nava Rubin

Research output: Contribution to journalArticle

Abstract

Much work has been done on the question of how the visual system extracts the three-dimensional (3D) structure and motion of an object from two-dimensional (2D) motion information, a problem known as 'Structure from Motion', or SFM. Much less is known, however, about the human ability to recover structure and motion when the optic flow field arises from multiple objects, although observations of this ability date as early as Ullman's well-known two-cylinders stimulus [The interpretation of visual motion (1979)]. In the presence of multiple objects, the SFM problem is further aggravated by the need to solve the segmentation problem, i.e. deciding which motion signal belongs to which object. Here, we present a model for how the human visual system solves the combined SFM and segmentation problems, which we term SSFM, concurrently. The model is based on computation of a simple scalar property of the optic flow field known as def, which was previously shown to be used by human observers in SFM. The def values of many triplets of moving dots are computed, and the identification of multiple objects the image is based on detecting multiple peaks in the histogram of def values. In five experiments, we show that human SSFM performance is consistent with the predictions of the model. We compare the predictions of our model to those of other theoretical approaches, in particular those that use a rigidity hypothesis, and discuss the validity of each approach as a model for human SSFM.

Original languageEnglish (US)
Pages (from-to)2715-2732
Number of pages18
JournalVision Research
Volume41
Issue number21
DOIs
StatePublished - 2001

Fingerprint

Psychophysics
Optic Flow
Aptitude

Keywords

  • Motion
  • Perceptual segmentation
  • Structure
  • Structure from motion

ASJC Scopus subject areas

  • Ophthalmology
  • Sensory Systems

Cite this

Segmentation in structure from motion : Modeling and psychophysics. / Caudek, Corrado; Rubin, Nava.

In: Vision Research, Vol. 41, No. 21, 2001, p. 2715-2732.

Research output: Contribution to journalArticle

Caudek, Corrado ; Rubin, Nava. / Segmentation in structure from motion : Modeling and psychophysics. In: Vision Research. 2001 ; Vol. 41, No. 21. pp. 2715-2732.
@article{50f36ab06ff04607860206c5aeb75726,
title = "Segmentation in structure from motion: Modeling and psychophysics",
abstract = "Much work has been done on the question of how the visual system extracts the three-dimensional (3D) structure and motion of an object from two-dimensional (2D) motion information, a problem known as 'Structure from Motion', or SFM. Much less is known, however, about the human ability to recover structure and motion when the optic flow field arises from multiple objects, although observations of this ability date as early as Ullman's well-known two-cylinders stimulus [The interpretation of visual motion (1979)]. In the presence of multiple objects, the SFM problem is further aggravated by the need to solve the segmentation problem, i.e. deciding which motion signal belongs to which object. Here, we present a model for how the human visual system solves the combined SFM and segmentation problems, which we term SSFM, concurrently. The model is based on computation of a simple scalar property of the optic flow field known as def, which was previously shown to be used by human observers in SFM. The def values of many triplets of moving dots are computed, and the identification of multiple objects the image is based on detecting multiple peaks in the histogram of def values. In five experiments, we show that human SSFM performance is consistent with the predictions of the model. We compare the predictions of our model to those of other theoretical approaches, in particular those that use a rigidity hypothesis, and discuss the validity of each approach as a model for human SSFM.",
keywords = "Motion, Perceptual segmentation, Structure, Structure from motion",
author = "Corrado Caudek and Nava Rubin",
year = "2001",
doi = "10.1016/S0042-6989(01)00163-8",
language = "English (US)",
volume = "41",
pages = "2715--2732",
journal = "Vision Research",
issn = "0042-6989",
publisher = "Elsevier Limited",
number = "21",

}

TY - JOUR

T1 - Segmentation in structure from motion

T2 - Modeling and psychophysics

AU - Caudek, Corrado

AU - Rubin, Nava

PY - 2001

Y1 - 2001

N2 - Much work has been done on the question of how the visual system extracts the three-dimensional (3D) structure and motion of an object from two-dimensional (2D) motion information, a problem known as 'Structure from Motion', or SFM. Much less is known, however, about the human ability to recover structure and motion when the optic flow field arises from multiple objects, although observations of this ability date as early as Ullman's well-known two-cylinders stimulus [The interpretation of visual motion (1979)]. In the presence of multiple objects, the SFM problem is further aggravated by the need to solve the segmentation problem, i.e. deciding which motion signal belongs to which object. Here, we present a model for how the human visual system solves the combined SFM and segmentation problems, which we term SSFM, concurrently. The model is based on computation of a simple scalar property of the optic flow field known as def, which was previously shown to be used by human observers in SFM. The def values of many triplets of moving dots are computed, and the identification of multiple objects the image is based on detecting multiple peaks in the histogram of def values. In five experiments, we show that human SSFM performance is consistent with the predictions of the model. We compare the predictions of our model to those of other theoretical approaches, in particular those that use a rigidity hypothesis, and discuss the validity of each approach as a model for human SSFM.

AB - Much work has been done on the question of how the visual system extracts the three-dimensional (3D) structure and motion of an object from two-dimensional (2D) motion information, a problem known as 'Structure from Motion', or SFM. Much less is known, however, about the human ability to recover structure and motion when the optic flow field arises from multiple objects, although observations of this ability date as early as Ullman's well-known two-cylinders stimulus [The interpretation of visual motion (1979)]. In the presence of multiple objects, the SFM problem is further aggravated by the need to solve the segmentation problem, i.e. deciding which motion signal belongs to which object. Here, we present a model for how the human visual system solves the combined SFM and segmentation problems, which we term SSFM, concurrently. The model is based on computation of a simple scalar property of the optic flow field known as def, which was previously shown to be used by human observers in SFM. The def values of many triplets of moving dots are computed, and the identification of multiple objects the image is based on detecting multiple peaks in the histogram of def values. In five experiments, we show that human SSFM performance is consistent with the predictions of the model. We compare the predictions of our model to those of other theoretical approaches, in particular those that use a rigidity hypothesis, and discuss the validity of each approach as a model for human SSFM.

KW - Motion

KW - Perceptual segmentation

KW - Structure

KW - Structure from motion

UR - http://www.scopus.com/inward/record.url?scp=0034824416&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034824416&partnerID=8YFLogxK

U2 - 10.1016/S0042-6989(01)00163-8

DO - 10.1016/S0042-6989(01)00163-8

M3 - Article

VL - 41

SP - 2715

EP - 2732

JO - Vision Research

JF - Vision Research

SN - 0042-6989

IS - 21

ER -