Image and depth from a conventional camera with a coded aperture

Anat Levin, Robert Fergus, Frédo Durand, William T. Freeman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint.

Original languageEnglish (US)
Title of host publicationProceedings of ACM SIGGRAPH 2007
DOIs
StatePublished - 2007
Event34th Annual Meeting of the Association for Computing Machinery's Special Interest Group on Graphics - San Diego, CA, United States
Duration: Aug 5 2007Aug 9 2007

Other

Other34th Annual Meeting of the Association for Computing Machinery's Special Interest Group on Graphics
CountryUnited States
CitySan Diego, CA
Period8/5/078/9/07

Fingerprint

Cameras
Camera lenses
Image resolution
Hardware
Recovery

Keywords

  • Coded imaging
  • Computational photography
  • Deblurring
  • Depth of field
  • Image statistics
  • Range estimation

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Levin, A., Fergus, R., Durand, F., & Freeman, W. T. (2007). Image and depth from a conventional camera with a coded aperture. In Proceedings of ACM SIGGRAPH 2007 [70] https://doi.org/10.1145/1275808.1276464

Image and depth from a conventional camera with a coded aperture. / Levin, Anat; Fergus, Robert; Durand, Frédo; Freeman, William T.

Proceedings of ACM SIGGRAPH 2007. 2007. 70.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Levin, A, Fergus, R, Durand, F & Freeman, WT 2007, Image and depth from a conventional camera with a coded aperture. in Proceedings of ACM SIGGRAPH 2007., 70, 34th Annual Meeting of the Association for Computing Machinery's Special Interest Group on Graphics, San Diego, CA, United States, 8/5/07. https://doi.org/10.1145/1275808.1276464
Levin A, Fergus R, Durand F, Freeman WT. Image and depth from a conventional camera with a coded aperture. In Proceedings of ACM SIGGRAPH 2007. 2007. 70 https://doi.org/10.1145/1275808.1276464
Levin, Anat ; Fergus, Robert ; Durand, Frédo ; Freeman, William T. / Image and depth from a conventional camera with a coded aperture. Proceedings of ACM SIGGRAPH 2007. 2007.
@inproceedings{af097afcaccd4e2980929ff466851503,
title = "Image and depth from a conventional camera with a coded aperture",
abstract = "A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint.",
keywords = "Coded imaging, Computational photography, Deblurring, Depth of field, Image statistics, Range estimation",
author = "Anat Levin and Robert Fergus and Fr{\'e}do Durand and Freeman, {William T.}",
year = "2007",
doi = "10.1145/1275808.1276464",
language = "English (US)",
booktitle = "Proceedings of ACM SIGGRAPH 2007",

}

TY - GEN

T1 - Image and depth from a conventional camera with a coded aperture

AU - Levin, Anat

AU - Fergus, Robert

AU - Durand, Frédo

AU - Freeman, William T.

PY - 2007

Y1 - 2007

N2 - A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint.

AB - A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint.

KW - Coded imaging

KW - Computational photography

KW - Deblurring

KW - Depth of field

KW - Image statistics

KW - Range estimation

UR - http://www.scopus.com/inward/record.url?scp=36949035987&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=36949035987&partnerID=8YFLogxK

U2 - 10.1145/1275808.1276464

DO - 10.1145/1275808.1276464

M3 - Conference contribution

AN - SCOPUS:36949035987

BT - Proceedings of ACM SIGGRAPH 2007

ER -