Perceptually optimized image rendering

Valero Laparra, Alexander Berardino, Johannes Ballé, Eero Simoncelli

Research output: Contribution to journalArticle

Abstract

We develop a framework for rendering photographic images by directly optimizing their perceptual similarity to the original visual scene. Specifically, over the set of all images that can be rendered on a given display, we minimize the normalized Laplacian pyramid distance (NLPD), a measure of perceptual dissimilarity that is derived from a simple model of the early stages of the human visual system. When rendering images acquired with a higher dynamic range than that of the display, we find that the optimization boosts the contrast of low-contrast features without introducing significant artifacts, yielding results of comparable visual quality to current state-of-the-art methods, but without manual intervention or parameter adjustment. We also demonstrate the effectiveness of the framework for a variety of other display constraints, including limitations on minimum luminance (black point), mean luminance (as a proxy for energy consumption), and quantized luminance levels (halftoning). We show that the method may generally be used to enhance details and contrast, and, in particular, can be used on images degraded by optical scattering (e.g., fog). Finally, we demonstrate the necessity of each of the NLPD components—an initial power function, a multiscale transform, and local contrast gain control—in achieving these results and we show that NLPD is competitive with the current state-of-the-art image quality metrics.

Original languageEnglish (US)
Pages (from-to)1511-1525
Number of pages15
JournalJournal of the Optical Society of America A: Optics and Image Science, and Vision
Volume34
Issue number9
DOIs
StatePublished - Sep 1 2017

Fingerprint

Luminance
Display devices
luminance
pyramids
Weather
Proxy
Artifacts
Fog
Image quality
Energy utilization
fog
Scattering
energy consumption
acceleration (physics)
dynamic range
artifacts
adjusting
optimization
scattering

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Computer Vision and Pattern Recognition

Cite this

Perceptually optimized image rendering. / Laparra, Valero; Berardino, Alexander; Ballé, Johannes; Simoncelli, Eero.

In: Journal of the Optical Society of America A: Optics and Image Science, and Vision, Vol. 34, No. 9, 01.09.2017, p. 1511-1525.

Research output: Contribution to journalArticle

Laparra, Valero ; Berardino, Alexander ; Ballé, Johannes ; Simoncelli, Eero. / Perceptually optimized image rendering. In: Journal of the Optical Society of America A: Optics and Image Science, and Vision. 2017 ; Vol. 34, No. 9. pp. 1511-1525.
@article{2e1ece6d62fb490aa5e0db69cc17a6fc,
title = "Perceptually optimized image rendering",
abstract = "We develop a framework for rendering photographic images by directly optimizing their perceptual similarity to the original visual scene. Specifically, over the set of all images that can be rendered on a given display, we minimize the normalized Laplacian pyramid distance (NLPD), a measure of perceptual dissimilarity that is derived from a simple model of the early stages of the human visual system. When rendering images acquired with a higher dynamic range than that of the display, we find that the optimization boosts the contrast of low-contrast features without introducing significant artifacts, yielding results of comparable visual quality to current state-of-the-art methods, but without manual intervention or parameter adjustment. We also demonstrate the effectiveness of the framework for a variety of other display constraints, including limitations on minimum luminance (black point), mean luminance (as a proxy for energy consumption), and quantized luminance levels (halftoning). We show that the method may generally be used to enhance details and contrast, and, in particular, can be used on images degraded by optical scattering (e.g., fog). Finally, we demonstrate the necessity of each of the NLPD components—an initial power function, a multiscale transform, and local contrast gain control—in achieving these results and we show that NLPD is competitive with the current state-of-the-art image quality metrics.",
author = "Valero Laparra and Alexander Berardino and Johannes Ball{\'e} and Eero Simoncelli",
year = "2017",
month = "9",
day = "1",
doi = "10.1364/JOSAA.34.001511",
language = "English (US)",
volume = "34",
pages = "1511--1525",
journal = "Journal of the Optical Society of America A: Optics and Image Science, and Vision",
issn = "0740-3232",
publisher = "The Optical Society",
number = "9",

}

TY - JOUR

T1 - Perceptually optimized image rendering

AU - Laparra, Valero

AU - Berardino, Alexander

AU - Ballé, Johannes

AU - Simoncelli, Eero

PY - 2017/9/1

Y1 - 2017/9/1

N2 - We develop a framework for rendering photographic images by directly optimizing their perceptual similarity to the original visual scene. Specifically, over the set of all images that can be rendered on a given display, we minimize the normalized Laplacian pyramid distance (NLPD), a measure of perceptual dissimilarity that is derived from a simple model of the early stages of the human visual system. When rendering images acquired with a higher dynamic range than that of the display, we find that the optimization boosts the contrast of low-contrast features without introducing significant artifacts, yielding results of comparable visual quality to current state-of-the-art methods, but without manual intervention or parameter adjustment. We also demonstrate the effectiveness of the framework for a variety of other display constraints, including limitations on minimum luminance (black point), mean luminance (as a proxy for energy consumption), and quantized luminance levels (halftoning). We show that the method may generally be used to enhance details and contrast, and, in particular, can be used on images degraded by optical scattering (e.g., fog). Finally, we demonstrate the necessity of each of the NLPD components—an initial power function, a multiscale transform, and local contrast gain control—in achieving these results and we show that NLPD is competitive with the current state-of-the-art image quality metrics.

AB - We develop a framework for rendering photographic images by directly optimizing their perceptual similarity to the original visual scene. Specifically, over the set of all images that can be rendered on a given display, we minimize the normalized Laplacian pyramid distance (NLPD), a measure of perceptual dissimilarity that is derived from a simple model of the early stages of the human visual system. When rendering images acquired with a higher dynamic range than that of the display, we find that the optimization boosts the contrast of low-contrast features without introducing significant artifacts, yielding results of comparable visual quality to current state-of-the-art methods, but without manual intervention or parameter adjustment. We also demonstrate the effectiveness of the framework for a variety of other display constraints, including limitations on minimum luminance (black point), mean luminance (as a proxy for energy consumption), and quantized luminance levels (halftoning). We show that the method may generally be used to enhance details and contrast, and, in particular, can be used on images degraded by optical scattering (e.g., fog). Finally, we demonstrate the necessity of each of the NLPD components—an initial power function, a multiscale transform, and local contrast gain control—in achieving these results and we show that NLPD is competitive with the current state-of-the-art image quality metrics.

UR - http://www.scopus.com/inward/record.url?scp=85028562363&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028562363&partnerID=8YFLogxK

U2 - 10.1364/JOSAA.34.001511

DO - 10.1364/JOSAA.34.001511

M3 - Article

VL - 34

SP - 1511

EP - 1525

JO - Journal of the Optical Society of America A: Optics and Image Science, and Vision

JF - Journal of the Optical Society of America A: Optics and Image Science, and Vision

SN - 0740-3232

IS - 9

ER -