Noisy depth maps fusion for multiview stereo via matrix completion

Yue Deng, Yebin Liu, Qionghai Dai, Zengke Zhang, Yao Wang

Research output: Contribution to journalArticle

Abstract

This paper introduces a general framework to fuse noisy point clouds from multiview images of the same object. We solve this classical vision problem using a newly emerging signal processing technique known as matrix completion. With this framework, we construct the initial incomplete matrix from the observed point clouds by all the cameras, with the invisible points by any camera denoted as unknown entries. The observed points corresponding to the same object point are put into the same row. When properly completed, the recovered matrix should have rank one, since all the columns describe the same object. Therefore, an intuitive approach to complete the matrix is by minimizing its rank subject to consistency with observed entries. In order to improve the fusion accuracy, we propose a general noisy matrix completion method called log-sum penalty completion (LPC), which is particularly effective in removing outliers. Based on the majorization-minimization algorithm (MM), the non-convex LPC problem is effectively solved by a sequence of convex optimizations. Experimental results on both point cloud fusion and MVS reconstructions verify the effectiveness of the proposed framework and the LPC algorithm.

Original languageEnglish (US)
Article number6187689
Pages (from-to)566-582
Number of pages17
JournalIEEE Journal on Selected Topics in Signal Processing
Volume6
Issue number5
DOIs
StatePublished - 2012

Fingerprint

Fusion reactions
Cameras
Convex optimization
Electric fuses
Signal processing

Keywords

  • Compressive sensing
  • fusion
  • matrix completion
  • multiview stereo (MVS)
  • point cloud

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Signal Processing

Cite this

Noisy depth maps fusion for multiview stereo via matrix completion. / Deng, Yue; Liu, Yebin; Dai, Qionghai; Zhang, Zengke; Wang, Yao.

In: IEEE Journal on Selected Topics in Signal Processing, Vol. 6, No. 5, 6187689, 2012, p. 566-582.

Research output: Contribution to journalArticle

Deng, Yue ; Liu, Yebin ; Dai, Qionghai ; Zhang, Zengke ; Wang, Yao. / Noisy depth maps fusion for multiview stereo via matrix completion. In: IEEE Journal on Selected Topics in Signal Processing. 2012 ; Vol. 6, No. 5. pp. 566-582.
@article{560816b8188c497bb3b30cd53baf770c,
title = "Noisy depth maps fusion for multiview stereo via matrix completion",
abstract = "This paper introduces a general framework to fuse noisy point clouds from multiview images of the same object. We solve this classical vision problem using a newly emerging signal processing technique known as matrix completion. With this framework, we construct the initial incomplete matrix from the observed point clouds by all the cameras, with the invisible points by any camera denoted as unknown entries. The observed points corresponding to the same object point are put into the same row. When properly completed, the recovered matrix should have rank one, since all the columns describe the same object. Therefore, an intuitive approach to complete the matrix is by minimizing its rank subject to consistency with observed entries. In order to improve the fusion accuracy, we propose a general noisy matrix completion method called log-sum penalty completion (LPC), which is particularly effective in removing outliers. Based on the majorization-minimization algorithm (MM), the non-convex LPC problem is effectively solved by a sequence of convex optimizations. Experimental results on both point cloud fusion and MVS reconstructions verify the effectiveness of the proposed framework and the LPC algorithm.",
keywords = "Compressive sensing, fusion, matrix completion, multiview stereo (MVS), point cloud",
author = "Yue Deng and Yebin Liu and Qionghai Dai and Zengke Zhang and Yao Wang",
year = "2012",
doi = "10.1109/JSTSP.2012.2195472",
language = "English (US)",
volume = "6",
pages = "566--582",
journal = "IEEE Journal on Selected Topics in Signal Processing",
issn = "1932-4553",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "5",

}

TY - JOUR

T1 - Noisy depth maps fusion for multiview stereo via matrix completion

AU - Deng, Yue

AU - Liu, Yebin

AU - Dai, Qionghai

AU - Zhang, Zengke

AU - Wang, Yao

PY - 2012

Y1 - 2012

N2 - This paper introduces a general framework to fuse noisy point clouds from multiview images of the same object. We solve this classical vision problem using a newly emerging signal processing technique known as matrix completion. With this framework, we construct the initial incomplete matrix from the observed point clouds by all the cameras, with the invisible points by any camera denoted as unknown entries. The observed points corresponding to the same object point are put into the same row. When properly completed, the recovered matrix should have rank one, since all the columns describe the same object. Therefore, an intuitive approach to complete the matrix is by minimizing its rank subject to consistency with observed entries. In order to improve the fusion accuracy, we propose a general noisy matrix completion method called log-sum penalty completion (LPC), which is particularly effective in removing outliers. Based on the majorization-minimization algorithm (MM), the non-convex LPC problem is effectively solved by a sequence of convex optimizations. Experimental results on both point cloud fusion and MVS reconstructions verify the effectiveness of the proposed framework and the LPC algorithm.

AB - This paper introduces a general framework to fuse noisy point clouds from multiview images of the same object. We solve this classical vision problem using a newly emerging signal processing technique known as matrix completion. With this framework, we construct the initial incomplete matrix from the observed point clouds by all the cameras, with the invisible points by any camera denoted as unknown entries. The observed points corresponding to the same object point are put into the same row. When properly completed, the recovered matrix should have rank one, since all the columns describe the same object. Therefore, an intuitive approach to complete the matrix is by minimizing its rank subject to consistency with observed entries. In order to improve the fusion accuracy, we propose a general noisy matrix completion method called log-sum penalty completion (LPC), which is particularly effective in removing outliers. Based on the majorization-minimization algorithm (MM), the non-convex LPC problem is effectively solved by a sequence of convex optimizations. Experimental results on both point cloud fusion and MVS reconstructions verify the effectiveness of the proposed framework and the LPC algorithm.

KW - Compressive sensing

KW - fusion

KW - matrix completion

KW - multiview stereo (MVS)

KW - point cloud

UR - http://www.scopus.com/inward/record.url?scp=84865417152&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84865417152&partnerID=8YFLogxK

U2 - 10.1109/JSTSP.2012.2195472

DO - 10.1109/JSTSP.2012.2195472

M3 - Article

VL - 6

SP - 566

EP - 582

JO - IEEE Journal on Selected Topics in Signal Processing

JF - IEEE Journal on Selected Topics in Signal Processing

SN - 1932-4553

IS - 5

M1 - 6187689

ER -