GIFS: Neural Implicit Function
for General Shape Representation

CVPR 2022

Jianglong Ye1, Yuntao Chen2, Naiyan Wang2, Xiaolong Wang1
1UC San Diego, 2TuSimple
Paper arXiv Video Slide Code

GIFS allows the reconstruction of general object shapes including
watertight, non-watertight, and multi-layer shapes.

Abstract

Recent development of neural implicit function has shown tremendous success on high-quality 3D shape reconstruction. However, most works divide the space into inside and outside of the shape, which limits their representing power to single-layer and watertight shapes. This limitation leads to tedious data processing (converting non-watertight raw data to watertight) as well as the incapability of representing general object shapes in the real world.

In this work, we propose a novel method to represent general shapes including non-watertight shapes and shapes with multi-layer surfaces. We introduce General Implicit Function for 3D Shape (GIFS), which models the relationships between every two points instead of the relationships between points and surfaces. Instead of dividing 3D space into predefined inside-outside regions, GIFS encodes whether two points are separated by any surface. Experiments on ShapeNet show that GIFS outperforms previous state-of-the-art methods in terms of reconstruction quality, rendering efficiency, and visual fidelity.

Video

GIFS Architecture

We learn a neural implicit function to classify whether two points are on the same side of object surfaces. First, the input point cloud is encoded to grid features. Given two 3D points, corresponding features are extracted from the grid. Then a maxpool operator is utilized to keep the permutation invariance. The decoder takes the fused feature as input and approximates the binary flag between points. An extra UDF branch can be used to enhance the spatial perception of the feature.

Reconstruction Results

Multi-layer Shapes

Our method can reconstruct internal structures of various shapes.


Non-watertight Shapes

The non-watertight shapes are difficult for traditional implicit neural representations to reconstruct.


Garments

Our method allows the reconstruction of the non-watertight garments.


Comparison

General Shapes

Our method can reconstruct smooth, continuous surfaces, and achieve a better visual effect than previous methods.

comparison-watertight

Watertight Shapes

Our method can reconstruct watertight shapes with the same accuracy as the state-of-the-art method.

comparison-watertight

BibTeX

@inproceedings{ye2022gifs,
  title={GIFS: Neural Implicit Function for General Shape Representation},
  author={Ye, Jianglong and Chen, Yuntao and Wang, Naiyan and Wang, Xiaolong},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2022}
}