PC-Net: Unsupervised Point Correspondence Learning with Neural Networks

Xiang Li, Lingjing Wang, Yi Fang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Point sets correspondence concerns with the establishment of point-wise correspondence for a group of 2D or 3D point sets with similar shape description. Existing methods often iteratively search for the optimal point-wise correspondence assignment for two sets of points, driven by maximizing the similarity between two sets of explicitly designed point features or by determining the parametric transformation for the best alignment between two point sets. In contrast, without depending on the explicit definitions of point features or transformation, our paper introduces a novel point correspondence neural networks (PC-Net) that is able to learn and predict the point correspondence among the populations of a specific object (e.g. fish, human, chair, etc) in an unsupervised manner. Specifically, in this paper, we first develop an encoder to learn the shape descriptor from a point set that captures essential global and deformation-insensitive geometric properties. Then followed with a novel motion-driven process, our PC-Net drives a template shape, that consists of a set of landmark points, morph and conform around a target shape object which is reconstructed through decoding the previously characterized shape descriptor. As a result, the motion-driven process progressively and coherently drifts all landmark points from the template shape to corresponding positions on the target object shape. The experimental results demonstrate that PC-Net can establish robust unsupervised point correspondence over a group of deformable object shapes in the presence of geometric noise and missing points. More importantly, with great generalization capability, PC-Net is capable of instantly predicting group point corresponding for unseen point sets.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 International Conference on 3D Vision, 3DV 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages145-154
Number of pages10
ISBN (Electronic)9781728131313
DOIs
StatePublished - Sep 2019
Event7th International Conference on 3D Vision, 3DV 2019 - Quebec, Canada
Duration: Sep 15 2019Sep 18 2019

Publication series

NameProceedings - 2019 International Conference on 3D Vision, 3DV 2019

Conference

Conference7th International Conference on 3D Vision, 3DV 2019
CountryCanada
CityQuebec
Period9/15/199/18/19

Fingerprint

Correspondence
Neural Networks
Neural networks
Point Sets
Point groups
Fish
Shape Descriptor
Decoding
Feature Point
Landmarks
Template
Learning
Deformable Objects
Target
Motion
Point Process
Encoder
Set of points
Alignment
Assignment

Keywords

  • correspondence
  • landmark
  • point cloud
  • Unsupervised learning

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Media Technology
  • Modeling and Simulation

Cite this

Li, X., Wang, L., & Fang, Y. (2019). PC-Net: Unsupervised Point Correspondence Learning with Neural Networks. In Proceedings - 2019 International Conference on 3D Vision, 3DV 2019 (pp. 145-154). [8885653] (Proceedings - 2019 International Conference on 3D Vision, 3DV 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/3DV.2019.00025

PC-Net : Unsupervised Point Correspondence Learning with Neural Networks. / Li, Xiang; Wang, Lingjing; Fang, Yi.

Proceedings - 2019 International Conference on 3D Vision, 3DV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 145-154 8885653 (Proceedings - 2019 International Conference on 3D Vision, 3DV 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Li, X, Wang, L & Fang, Y 2019, PC-Net: Unsupervised Point Correspondence Learning with Neural Networks. in Proceedings - 2019 International Conference on 3D Vision, 3DV 2019., 8885653, Proceedings - 2019 International Conference on 3D Vision, 3DV 2019, Institute of Electrical and Electronics Engineers Inc., pp. 145-154, 7th International Conference on 3D Vision, 3DV 2019, Quebec, Canada, 9/15/19. https://doi.org/10.1109/3DV.2019.00025
Li X, Wang L, Fang Y. PC-Net: Unsupervised Point Correspondence Learning with Neural Networks. In Proceedings - 2019 International Conference on 3D Vision, 3DV 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 145-154. 8885653. (Proceedings - 2019 International Conference on 3D Vision, 3DV 2019). https://doi.org/10.1109/3DV.2019.00025
Li, Xiang ; Wang, Lingjing ; Fang, Yi. / PC-Net : Unsupervised Point Correspondence Learning with Neural Networks. Proceedings - 2019 International Conference on 3D Vision, 3DV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 145-154 (Proceedings - 2019 International Conference on 3D Vision, 3DV 2019).
@inproceedings{78651514b0624bd99b40b283aedacc60,
title = "PC-Net: Unsupervised Point Correspondence Learning with Neural Networks",
abstract = "Point sets correspondence concerns with the establishment of point-wise correspondence for a group of 2D or 3D point sets with similar shape description. Existing methods often iteratively search for the optimal point-wise correspondence assignment for two sets of points, driven by maximizing the similarity between two sets of explicitly designed point features or by determining the parametric transformation for the best alignment between two point sets. In contrast, without depending on the explicit definitions of point features or transformation, our paper introduces a novel point correspondence neural networks (PC-Net) that is able to learn and predict the point correspondence among the populations of a specific object (e.g. fish, human, chair, etc) in an unsupervised manner. Specifically, in this paper, we first develop an encoder to learn the shape descriptor from a point set that captures essential global and deformation-insensitive geometric properties. Then followed with a novel motion-driven process, our PC-Net drives a template shape, that consists of a set of landmark points, morph and conform around a target shape object which is reconstructed through decoding the previously characterized shape descriptor. As a result, the motion-driven process progressively and coherently drifts all landmark points from the template shape to corresponding positions on the target object shape. The experimental results demonstrate that PC-Net can establish robust unsupervised point correspondence over a group of deformable object shapes in the presence of geometric noise and missing points. More importantly, with great generalization capability, PC-Net is capable of instantly predicting group point corresponding for unseen point sets.",
keywords = "correspondence, landmark, point cloud, Unsupervised learning",
author = "Xiang Li and Lingjing Wang and Yi Fang",
year = "2019",
month = "9",
doi = "10.1109/3DV.2019.00025",
language = "English (US)",
series = "Proceedings - 2019 International Conference on 3D Vision, 3DV 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "145--154",
booktitle = "Proceedings - 2019 International Conference on 3D Vision, 3DV 2019",

}

TY - GEN

T1 - PC-Net

T2 - Unsupervised Point Correspondence Learning with Neural Networks

AU - Li, Xiang

AU - Wang, Lingjing

AU - Fang, Yi

PY - 2019/9

Y1 - 2019/9

N2 - Point sets correspondence concerns with the establishment of point-wise correspondence for a group of 2D or 3D point sets with similar shape description. Existing methods often iteratively search for the optimal point-wise correspondence assignment for two sets of points, driven by maximizing the similarity between two sets of explicitly designed point features or by determining the parametric transformation for the best alignment between two point sets. In contrast, without depending on the explicit definitions of point features or transformation, our paper introduces a novel point correspondence neural networks (PC-Net) that is able to learn and predict the point correspondence among the populations of a specific object (e.g. fish, human, chair, etc) in an unsupervised manner. Specifically, in this paper, we first develop an encoder to learn the shape descriptor from a point set that captures essential global and deformation-insensitive geometric properties. Then followed with a novel motion-driven process, our PC-Net drives a template shape, that consists of a set of landmark points, morph and conform around a target shape object which is reconstructed through decoding the previously characterized shape descriptor. As a result, the motion-driven process progressively and coherently drifts all landmark points from the template shape to corresponding positions on the target object shape. The experimental results demonstrate that PC-Net can establish robust unsupervised point correspondence over a group of deformable object shapes in the presence of geometric noise and missing points. More importantly, with great generalization capability, PC-Net is capable of instantly predicting group point corresponding for unseen point sets.

AB - Point sets correspondence concerns with the establishment of point-wise correspondence for a group of 2D or 3D point sets with similar shape description. Existing methods often iteratively search for the optimal point-wise correspondence assignment for two sets of points, driven by maximizing the similarity between two sets of explicitly designed point features or by determining the parametric transformation for the best alignment between two point sets. In contrast, without depending on the explicit definitions of point features or transformation, our paper introduces a novel point correspondence neural networks (PC-Net) that is able to learn and predict the point correspondence among the populations of a specific object (e.g. fish, human, chair, etc) in an unsupervised manner. Specifically, in this paper, we first develop an encoder to learn the shape descriptor from a point set that captures essential global and deformation-insensitive geometric properties. Then followed with a novel motion-driven process, our PC-Net drives a template shape, that consists of a set of landmark points, morph and conform around a target shape object which is reconstructed through decoding the previously characterized shape descriptor. As a result, the motion-driven process progressively and coherently drifts all landmark points from the template shape to corresponding positions on the target object shape. The experimental results demonstrate that PC-Net can establish robust unsupervised point correspondence over a group of deformable object shapes in the presence of geometric noise and missing points. More importantly, with great generalization capability, PC-Net is capable of instantly predicting group point corresponding for unseen point sets.

KW - correspondence

KW - landmark

KW - point cloud

KW - Unsupervised learning

UR - http://www.scopus.com/inward/record.url?scp=85074976765&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074976765&partnerID=8YFLogxK

U2 - 10.1109/3DV.2019.00025

DO - 10.1109/3DV.2019.00025

M3 - Conference contribution

AN - SCOPUS:85074976765

T3 - Proceedings - 2019 International Conference on 3D Vision, 3DV 2019

SP - 145

EP - 154

BT - Proceedings - 2019 International Conference on 3D Vision, 3DV 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -