Restoring an image taken through a window covered with dirt or rain

David Eigen, Dilip Krishnan, Rob Fergus

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Photographs taken through a window are often compromised by dirt or rain present on the window surface. Common cases of this include pictures taken from inside a vehicle, or outdoor security cameras mounted inside a protective enclosure. At capture time, defocus can be used to remove the artifacts, but this relies on achieving a shallow depth-of-field and placement of the camera close to the window. Instead, we present a post-capture image processing solution that can remove localized rain and dirt artifacts from a single image. We collect a dataset of clean/corrupted image pairs which are then used to train a specialized form of convolutional neural network. This learns how to map corrupted image patches to clean ones, implicitly capturing the characteristic appearance of dirt and water droplets in natural images. Our models demonstrate effective removal of dirt and rain in outdoor test conditions.

Original languageEnglish (US)
Title of host publicationProceedings - 2013 IEEE International Conference on Computer Vision, ICCV 2013
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages633-640
Number of pages8
ISBN (Print)9781479928392
DOIs
StatePublished - 2013
Event2013 14th IEEE International Conference on Computer Vision, ICCV 2013 - Sydney, NSW, Australia
Duration: Dec 1 2013Dec 8 2013

Other

Other2013 14th IEEE International Conference on Computer Vision, ICCV 2013
CountryAustralia
CitySydney, NSW
Period12/1/1312/8/13

Fingerprint

Rain
Cameras
Enclosures
Image processing
Neural networks
Water

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Eigen, D., Krishnan, D., & Fergus, R. (2013). Restoring an image taken through a window covered with dirt or rain. In Proceedings - 2013 IEEE International Conference on Computer Vision, ICCV 2013 (pp. 633-640). [6751188] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2013.84

Restoring an image taken through a window covered with dirt or rain. / Eigen, David; Krishnan, Dilip; Fergus, Rob.

Proceedings - 2013 IEEE International Conference on Computer Vision, ICCV 2013. Institute of Electrical and Electronics Engineers Inc., 2013. p. 633-640 6751188.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Eigen, D, Krishnan, D & Fergus, R 2013, Restoring an image taken through a window covered with dirt or rain. in Proceedings - 2013 IEEE International Conference on Computer Vision, ICCV 2013., 6751188, Institute of Electrical and Electronics Engineers Inc., pp. 633-640, 2013 14th IEEE International Conference on Computer Vision, ICCV 2013, Sydney, NSW, Australia, 12/1/13. https://doi.org/10.1109/ICCV.2013.84
Eigen D, Krishnan D, Fergus R. Restoring an image taken through a window covered with dirt or rain. In Proceedings - 2013 IEEE International Conference on Computer Vision, ICCV 2013. Institute of Electrical and Electronics Engineers Inc. 2013. p. 633-640. 6751188 https://doi.org/10.1109/ICCV.2013.84
Eigen, David ; Krishnan, Dilip ; Fergus, Rob. / Restoring an image taken through a window covered with dirt or rain. Proceedings - 2013 IEEE International Conference on Computer Vision, ICCV 2013. Institute of Electrical and Electronics Engineers Inc., 2013. pp. 633-640
@inproceedings{a449772a572946ca9e7266e91b52751b,
title = "Restoring an image taken through a window covered with dirt or rain",
abstract = "Photographs taken through a window are often compromised by dirt or rain present on the window surface. Common cases of this include pictures taken from inside a vehicle, or outdoor security cameras mounted inside a protective enclosure. At capture time, defocus can be used to remove the artifacts, but this relies on achieving a shallow depth-of-field and placement of the camera close to the window. Instead, we present a post-capture image processing solution that can remove localized rain and dirt artifacts from a single image. We collect a dataset of clean/corrupted image pairs which are then used to train a specialized form of convolutional neural network. This learns how to map corrupted image patches to clean ones, implicitly capturing the characteristic appearance of dirt and water droplets in natural images. Our models demonstrate effective removal of dirt and rain in outdoor test conditions.",
author = "David Eigen and Dilip Krishnan and Rob Fergus",
year = "2013",
doi = "10.1109/ICCV.2013.84",
language = "English (US)",
isbn = "9781479928392",
pages = "633--640",
booktitle = "Proceedings - 2013 IEEE International Conference on Computer Vision, ICCV 2013",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Restoring an image taken through a window covered with dirt or rain

AU - Eigen, David

AU - Krishnan, Dilip

AU - Fergus, Rob

PY - 2013

Y1 - 2013

N2 - Photographs taken through a window are often compromised by dirt or rain present on the window surface. Common cases of this include pictures taken from inside a vehicle, or outdoor security cameras mounted inside a protective enclosure. At capture time, defocus can be used to remove the artifacts, but this relies on achieving a shallow depth-of-field and placement of the camera close to the window. Instead, we present a post-capture image processing solution that can remove localized rain and dirt artifacts from a single image. We collect a dataset of clean/corrupted image pairs which are then used to train a specialized form of convolutional neural network. This learns how to map corrupted image patches to clean ones, implicitly capturing the characteristic appearance of dirt and water droplets in natural images. Our models demonstrate effective removal of dirt and rain in outdoor test conditions.

AB - Photographs taken through a window are often compromised by dirt or rain present on the window surface. Common cases of this include pictures taken from inside a vehicle, or outdoor security cameras mounted inside a protective enclosure. At capture time, defocus can be used to remove the artifacts, but this relies on achieving a shallow depth-of-field and placement of the camera close to the window. Instead, we present a post-capture image processing solution that can remove localized rain and dirt artifacts from a single image. We collect a dataset of clean/corrupted image pairs which are then used to train a specialized form of convolutional neural network. This learns how to map corrupted image patches to clean ones, implicitly capturing the characteristic appearance of dirt and water droplets in natural images. Our models demonstrate effective removal of dirt and rain in outdoor test conditions.

UR - http://www.scopus.com/inward/record.url?scp=84898798806&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898798806&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2013.84

DO - 10.1109/ICCV.2013.84

M3 - Conference contribution

AN - SCOPUS:84898798806

SN - 9781479928392

SP - 633

EP - 640

BT - Proceedings - 2013 IEEE International Conference on Computer Vision, ICCV 2013

PB - Institute of Electrical and Electronics Engineers Inc.

ER -