Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Learnable Triangulation for Deep Learning-based 3D Reconstruction of Objects of Arbitrary Topology from Single RGB Images

Abstract : We propose a novel deep reinforcement learning-based approach for 3D object reconstruction from monocular images. Prior works that use mesh representations are template based. Thus, they are limited to the reconstruction of objects that have the same topology as the template. Methods that use volumetric grids as intermediate representations are computationally expensive, which limits their application in real-time scenarios. In this paper, we propose a novel end-to-end method that reconstructs 3D objects of arbitrary topology from a monocular image. It is composed of of (1) a Vertex Generation Network (VGN), which predicts the initial 3D locations of the object's vertices from an input RGB image, (2) a differentiable triangulation layer, which learns in a non-supervised manner, using a novel reinforcement learning algorithm, the best triangulation of the object's vertices, and finally, (3) a hierarchical mesh refinement network that uses graph convolutions to refine the initial mesh. Our key contribution is the learnable triangulation process, which recovers in an unsupervised manner the topology of the input shape. Our experiments on ShapeNet and Pix3D benchmarks show that the proposed method outperforms the state-of-the-art in terms of visual quality, reconstruction accuracy, and computational time.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal-univ-orleans.archives-ouvertes.fr/hal-03553564
Contributor : Aladine Chetouani Connect in order to contact the contributor
Submitted on : Wednesday, February 2, 2022 - 8:17:40 PM
Last modification on : Thursday, February 3, 2022 - 3:28:09 AM

Links full text

Identifiers

  • HAL Id : hal-03553564, version 1
  • ARXIV : 2109.11844

Citation

Tarek Ben Charrada, Hedi Tabia, Aladine Chetouani, Hamid Laga. Learnable Triangulation for Deep Learning-based 3D Reconstruction of Objects of Arbitrary Topology from Single RGB Images. 2022. ⟨hal-03553564⟩

Share

Metrics

Record views

13