Image and depth from a conventional camera with a coded aperture

Anat Levin, Robert Fergus, Frédo Durand, William T. Freeman

Research output: Contribution to journalArticle

Abstract

A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint.

Original languageEnglish (US)
Article number1276464
JournalACM Transactions on Graphics
Volume26
Issue number3
DOIs
StatePublished - Jul 29 2007

Fingerprint

Cameras
Camera lenses
Image resolution
Hardware
Recovery

Keywords

  • Coded imaging
  • Computational photography
  • Deblurring
  • Depth of field
  • Image statistics
  • Range estimation

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Image and depth from a conventional camera with a coded aperture. / Levin, Anat; Fergus, Robert; Durand, Frédo; Freeman, William T.

In: ACM Transactions on Graphics, Vol. 26, No. 3, 1276464, 29.07.2007.

Research output: Contribution to journalArticle

Levin, Anat ; Fergus, Robert ; Durand, Frédo ; Freeman, William T. / Image and depth from a conventional camera with a coded aperture. In: ACM Transactions on Graphics. 2007 ; Vol. 26, No. 3.
@article{209718e4f0584f00984eb800de4f2aeb,
title = "Image and depth from a conventional camera with a coded aperture",
abstract = "A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint.",
keywords = "Coded imaging, Computational photography, Deblurring, Depth of field, Image statistics, Range estimation",
author = "Anat Levin and Robert Fergus and Fr{\'e}do Durand and Freeman, {William T.}",
year = "2007",
month = "7",
day = "29",
doi = "10.1145/1276377.1276464",
language = "English (US)",
volume = "26",
journal = "ACM Transactions on Graphics",
issn = "0730-0301",
publisher = "Association for Computing Machinery (ACM)",
number = "3",

}

TY - JOUR

T1 - Image and depth from a conventional camera with a coded aperture

AU - Levin, Anat

AU - Fergus, Robert

AU - Durand, Frédo

AU - Freeman, William T.

PY - 2007/7/29

Y1 - 2007/7/29

N2 - A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint.

AB - A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint.

KW - Coded imaging

KW - Computational photography

KW - Deblurring

KW - Depth of field

KW - Image statistics

KW - Range estimation

UR - http://www.scopus.com/inward/record.url?scp=34547674830&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34547674830&partnerID=8YFLogxK

U2 - 10.1145/1276377.1276464

DO - 10.1145/1276377.1276464

M3 - Article

AN - SCOPUS:34547674830

VL - 26

JO - ACM Transactions on Graphics

JF - ACM Transactions on Graphics

SN - 0730-0301

IS - 3

M1 - 1276464

ER -