Learning domain-invariant feature for robust depth-image-based 3D shape retrieval

Jing Zhu, John Ross Rizzo, Yi Fang

Research output: Contribution to journalArticle

Abstract

In recent years, 3D shape retrieval has been garnering increased attention in a wide range of fields, including graphics, image processing and computer vision. Meanwhile, with the advances in depth sensing techniques, such as those used by the Kinect and 3D LiDAR device, depth images of 3D objects can be acquired conveniently, leading to rapid increases of depth image dataset. In this paper, different from most of the traditional cross-domain 3D shape retrieval approaches that focused on the RGB-D image-based or sketch-based shape retrieval, we aim to retrieve shapes based only on depth image queries. Specifically, we proposed to learn a robust domain-invariant representation between 3D shape and depth image domains by constructing a pair of discriminative neural networks, one for each domain. The two networks are connected by a loss function with constraints on both inter-class and intra-class margins, which minimizes the intra-class variance while maximizing the inter-class margin among data from the two domains (depth image and 3D shape). Our experiments on the NYU Depth V2 dataset (with Kinect-type noise) and two 3D shape (CAD model) datasets (SHREC 2014 and ModelNet) demonstrate that our proposed technique performs superiorly over existing state-of-the-art approaches on depth-image-based 3D shape retrieval task.

Original languageEnglish (US)
JournalPattern Recognition Letters
DOIs
StateAccepted/In press - Jan 1 2017

Fingerprint

Computer vision
Computer aided design
Image processing
Neural networks
Experiments

Keywords

  • 3D shape retrieval
  • Cross-domain
  • Depth images
  • Discriminative neural network

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Learning domain-invariant feature for robust depth-image-based 3D shape retrieval. / Zhu, Jing; Rizzo, John Ross; Fang, Yi.

In: Pattern Recognition Letters, 01.01.2017.

Research output: Contribution to journalArticle

@article{c5ccd25c17774534b64909b7b81c4919,
title = "Learning domain-invariant feature for robust depth-image-based 3D shape retrieval",
abstract = "In recent years, 3D shape retrieval has been garnering increased attention in a wide range of fields, including graphics, image processing and computer vision. Meanwhile, with the advances in depth sensing techniques, such as those used by the Kinect and 3D LiDAR device, depth images of 3D objects can be acquired conveniently, leading to rapid increases of depth image dataset. In this paper, different from most of the traditional cross-domain 3D shape retrieval approaches that focused on the RGB-D image-based or sketch-based shape retrieval, we aim to retrieve shapes based only on depth image queries. Specifically, we proposed to learn a robust domain-invariant representation between 3D shape and depth image domains by constructing a pair of discriminative neural networks, one for each domain. The two networks are connected by a loss function with constraints on both inter-class and intra-class margins, which minimizes the intra-class variance while maximizing the inter-class margin among data from the two domains (depth image and 3D shape). Our experiments on the NYU Depth V2 dataset (with Kinect-type noise) and two 3D shape (CAD model) datasets (SHREC 2014 and ModelNet) demonstrate that our proposed technique performs superiorly over existing state-of-the-art approaches on depth-image-based 3D shape retrieval task.",
keywords = "3D shape retrieval, Cross-domain, Depth images, Discriminative neural network",
author = "Jing Zhu and Rizzo, {John Ross} and Yi Fang",
year = "2017",
month = "1",
day = "1",
doi = "10.1016/j.patrec.2017.09.041",
language = "English (US)",
journal = "Pattern Recognition Letters",
issn = "0167-8655",
publisher = "Elsevier",

}

TY - JOUR

T1 - Learning domain-invariant feature for robust depth-image-based 3D shape retrieval

AU - Zhu, Jing

AU - Rizzo, John Ross

AU - Fang, Yi

PY - 2017/1/1

Y1 - 2017/1/1

N2 - In recent years, 3D shape retrieval has been garnering increased attention in a wide range of fields, including graphics, image processing and computer vision. Meanwhile, with the advances in depth sensing techniques, such as those used by the Kinect and 3D LiDAR device, depth images of 3D objects can be acquired conveniently, leading to rapid increases of depth image dataset. In this paper, different from most of the traditional cross-domain 3D shape retrieval approaches that focused on the RGB-D image-based or sketch-based shape retrieval, we aim to retrieve shapes based only on depth image queries. Specifically, we proposed to learn a robust domain-invariant representation between 3D shape and depth image domains by constructing a pair of discriminative neural networks, one for each domain. The two networks are connected by a loss function with constraints on both inter-class and intra-class margins, which minimizes the intra-class variance while maximizing the inter-class margin among data from the two domains (depth image and 3D shape). Our experiments on the NYU Depth V2 dataset (with Kinect-type noise) and two 3D shape (CAD model) datasets (SHREC 2014 and ModelNet) demonstrate that our proposed technique performs superiorly over existing state-of-the-art approaches on depth-image-based 3D shape retrieval task.

AB - In recent years, 3D shape retrieval has been garnering increased attention in a wide range of fields, including graphics, image processing and computer vision. Meanwhile, with the advances in depth sensing techniques, such as those used by the Kinect and 3D LiDAR device, depth images of 3D objects can be acquired conveniently, leading to rapid increases of depth image dataset. In this paper, different from most of the traditional cross-domain 3D shape retrieval approaches that focused on the RGB-D image-based or sketch-based shape retrieval, we aim to retrieve shapes based only on depth image queries. Specifically, we proposed to learn a robust domain-invariant representation between 3D shape and depth image domains by constructing a pair of discriminative neural networks, one for each domain. The two networks are connected by a loss function with constraints on both inter-class and intra-class margins, which minimizes the intra-class variance while maximizing the inter-class margin among data from the two domains (depth image and 3D shape). Our experiments on the NYU Depth V2 dataset (with Kinect-type noise) and two 3D shape (CAD model) datasets (SHREC 2014 and ModelNet) demonstrate that our proposed technique performs superiorly over existing state-of-the-art approaches on depth-image-based 3D shape retrieval task.

KW - 3D shape retrieval

KW - Cross-domain

KW - Depth images

KW - Discriminative neural network

UR - http://www.scopus.com/inward/record.url?scp=85030773595&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85030773595&partnerID=8YFLogxK

U2 - 10.1016/j.patrec.2017.09.041

DO - 10.1016/j.patrec.2017.09.041

M3 - Article

AN - SCOPUS:85030773595

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

SN - 0167-8655

ER -