Unsupervised 3D Part Decomposition via Leveraged Gaussian Splatting

1Seoul National University, 2Clova AI, NAVER

Our method unsupervisedly infers the 3D parts of articulated objects from monocular videos by using Gaussian Splatting methods.

Abstract

We propose a novel unsupervised method for motion-based 3D part decomposition of articulated objects with a single monocular video of a dynamic scene. In contrast to existing unsupervised methods relying on optical flow or tracking techniques, our approach addresses this problem without additional information by leveraging Gaussian splatting techniques. We generate a series of Gaussians from a monocular video and then analyze the relationship between the Gaussians to decompose the dynamic scene into motion-based parts. To decompose dynamic scenes consisting of articulated objects, we design an articulated deformation field suitable for the movement of articulated objects. And to effectively understand the relationships of Gaussians of different shapes, we propose a 3D reconstruction loss using 3D occupied voxel maps generated from the Gaussians. Experimental results demonstrate that our method outperforms existing approaches in terms of 3D part decomposition for articulated objects and demonstrates competitive image synthesis performance.

Video

D-NeRF Datasets

Custom Datasets

Training Process

Ablation Studies

We propose 3D reconstruction loss to enhance the decomposition performance. We append the decomposition results using only the photormetric loss except the proposed 3D reconstruction loss. We append the result when using the Chmafer distance loss instead of the voxel-based 3D reconstruction loss.