Learning adversarial 3D model generation with 2D image enhancer

Jing Zhu, Jin Xie, Yi Fang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recent advancements in generative adversarial nets (GANs) and volumetric convolutional neural networks (CNNs) enable generating 3D models from a probabilistic space. In this paper, we have developed a novel GAN-based deep neural network to obtain a better latent space for the generation of 3D models. In the proposed method, an enhancer neural network is introduced to extract information from other corresponding domains (e.g. image) to improve the performance of the 3D model generator, and the discriminative power of the unsupervised shape features learned from the 3D model discriminator. Specifically, we train the 3D generative adversarial networks on 3D volumetric models, and at the same time, the enhancer network learns image features from rendered images. Different from the traditional GAN architecture that uses uninformative random vectors as inputs, we feed the high-level image features learned from the enhancer into the 3D model generator for better training. The evaluations on two large-scale 3D model datasets, ShapeNet and ModelNet, demonstrate that our proposed method can not only generate high-quality 3D models, but also successfully learn discriminative shape representation for classification and retrieval without supervision.

Original languageEnglish (US)
Title of host publication32nd AAAI Conference on Artificial Intelligence, AAAI 2018
PublisherAAAI press
Pages7615-7622
Number of pages8
ISBN (Electronic)9781577358008
StatePublished - Jan 1 2018
Event32nd AAAI Conference on Artificial Intelligence, AAAI 2018 - New Orleans, United States
Duration: Feb 2 2018Feb 7 2018

Other

Other32nd AAAI Conference on Artificial Intelligence, AAAI 2018
CountryUnited States
CityNew Orleans
Period2/2/182/7/18

Fingerprint

Neural networks
Discriminators
Deep neural networks

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Zhu, J., Xie, J., & Fang, Y. (2018). Learning adversarial 3D model generation with 2D image enhancer. In 32nd AAAI Conference on Artificial Intelligence, AAAI 2018 (pp. 7615-7622). AAAI press.

Learning adversarial 3D model generation with 2D image enhancer. / Zhu, Jing; Xie, Jin; Fang, Yi.

32nd AAAI Conference on Artificial Intelligence, AAAI 2018. AAAI press, 2018. p. 7615-7622.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhu, J, Xie, J & Fang, Y 2018, Learning adversarial 3D model generation with 2D image enhancer. in 32nd AAAI Conference on Artificial Intelligence, AAAI 2018. AAAI press, pp. 7615-7622, 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, New Orleans, United States, 2/2/18.
Zhu J, Xie J, Fang Y. Learning adversarial 3D model generation with 2D image enhancer. In 32nd AAAI Conference on Artificial Intelligence, AAAI 2018. AAAI press. 2018. p. 7615-7622
Zhu, Jing ; Xie, Jin ; Fang, Yi. / Learning adversarial 3D model generation with 2D image enhancer. 32nd AAAI Conference on Artificial Intelligence, AAAI 2018. AAAI press, 2018. pp. 7615-7622
@inproceedings{48ad09b0cb1744bfa442464a5cf00694,
title = "Learning adversarial 3D model generation with 2D image enhancer",
abstract = "Recent advancements in generative adversarial nets (GANs) and volumetric convolutional neural networks (CNNs) enable generating 3D models from a probabilistic space. In this paper, we have developed a novel GAN-based deep neural network to obtain a better latent space for the generation of 3D models. In the proposed method, an enhancer neural network is introduced to extract information from other corresponding domains (e.g. image) to improve the performance of the 3D model generator, and the discriminative power of the unsupervised shape features learned from the 3D model discriminator. Specifically, we train the 3D generative adversarial networks on 3D volumetric models, and at the same time, the enhancer network learns image features from rendered images. Different from the traditional GAN architecture that uses uninformative random vectors as inputs, we feed the high-level image features learned from the enhancer into the 3D model generator for better training. The evaluations on two large-scale 3D model datasets, ShapeNet and ModelNet, demonstrate that our proposed method can not only generate high-quality 3D models, but also successfully learn discriminative shape representation for classification and retrieval without supervision.",
author = "Jing Zhu and Jin Xie and Yi Fang",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
pages = "7615--7622",
booktitle = "32nd AAAI Conference on Artificial Intelligence, AAAI 2018",
publisher = "AAAI press",

}

TY - GEN

T1 - Learning adversarial 3D model generation with 2D image enhancer

AU - Zhu, Jing

AU - Xie, Jin

AU - Fang, Yi

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Recent advancements in generative adversarial nets (GANs) and volumetric convolutional neural networks (CNNs) enable generating 3D models from a probabilistic space. In this paper, we have developed a novel GAN-based deep neural network to obtain a better latent space for the generation of 3D models. In the proposed method, an enhancer neural network is introduced to extract information from other corresponding domains (e.g. image) to improve the performance of the 3D model generator, and the discriminative power of the unsupervised shape features learned from the 3D model discriminator. Specifically, we train the 3D generative adversarial networks on 3D volumetric models, and at the same time, the enhancer network learns image features from rendered images. Different from the traditional GAN architecture that uses uninformative random vectors as inputs, we feed the high-level image features learned from the enhancer into the 3D model generator for better training. The evaluations on two large-scale 3D model datasets, ShapeNet and ModelNet, demonstrate that our proposed method can not only generate high-quality 3D models, but also successfully learn discriminative shape representation for classification and retrieval without supervision.

AB - Recent advancements in generative adversarial nets (GANs) and volumetric convolutional neural networks (CNNs) enable generating 3D models from a probabilistic space. In this paper, we have developed a novel GAN-based deep neural network to obtain a better latent space for the generation of 3D models. In the proposed method, an enhancer neural network is introduced to extract information from other corresponding domains (e.g. image) to improve the performance of the 3D model generator, and the discriminative power of the unsupervised shape features learned from the 3D model discriminator. Specifically, we train the 3D generative adversarial networks on 3D volumetric models, and at the same time, the enhancer network learns image features from rendered images. Different from the traditional GAN architecture that uses uninformative random vectors as inputs, we feed the high-level image features learned from the enhancer into the 3D model generator for better training. The evaluations on two large-scale 3D model datasets, ShapeNet and ModelNet, demonstrate that our proposed method can not only generate high-quality 3D models, but also successfully learn discriminative shape representation for classification and retrieval without supervision.

UR - http://www.scopus.com/inward/record.url?scp=85058245481&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058245481&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85058245481

SP - 7615

EP - 7622

BT - 32nd AAAI Conference on Artificial Intelligence, AAAI 2018

PB - AAAI press

ER -