POC-SLT: Partial Object Completion with SDF Latent Transformers

Authors: Faezeh Zakeri, Raphael Braun, Lukas Ruppert, and Hendrik P.A. Lensch

Conference: CRV 2025

Arxiv: Arxiv 2024

Code Repository: poc-slt

---

Abstract

3D geometric shape completion hinges on representation learning and a deep understanding of geometric data. Without profound insights into the three-dimensional nature of the data, this task remains unattainable. Our work addresses this challenge of 3D shape completion given partial observations by proposing a transformer operating on a latent space representing Signed Distance Fields (SDFs). Instead of a monolithic volume, the SDF of an object is partitioned into smaller high-resolution patches leading to a sequence of latent codes. The approach relies on a smooth latent space encoding learned via a variational autoencoder (VAE), trained on millions of 3D patches. We employ an efficient masked autoencoder transformer to complete partial sequences into comprehensive shapes in latent space. Our approach is extensively evaluated on partial observations from ShapeNet and the ABC dataset where only fractions of the objects are given. The proposed POC-SLT architecture compares favorably with several baseline state-of-the-art methods, demonstrating a significant improvement in 3D shape completion, both qualitatively and quantitatively.

---

Demo Gif

For high quality video, click here!

Project Structure

├── data/
├── docs/
├── src/
├── requirements.txt
└── README.md

📦 Model checkpoint

Shape Completion on Shapenet

Shape Completion on ABC

Patchwise Variational Autoencoder (P-VAE) on Shapenet

Shapenet Completion Initialization Model

ABC Completion Initialization Model

Dataset

Datasets for evaluation are available here:

Shapenet Test LMDB

ABC Test LMDB

P-VAE Test LMDB

Running Instructions

• Evaluation

  • Shapenet Completion Model

    • random partial eval_config_random_shapenet
    • masked partial eval_config_shapenet

  • ABC Completion Model

    • masked partial eval_config_abc

• Train

  • Shapenet Completion Model

    • with initialization model

      • train_no_empty_masking_custom_shapenet_config
      • initialization model is available at poc-slt-shapenet-completion-initialization-model

    • from scratch

      • you can use the code from train_no_empty_masking_custom_shapenet_config and remove the initialization from pre-trained model provided and train from scratch.

  • ABC Completion Model

    • with initialization model

      • train_no_empty_masking_custom_abc_config
      • initialization model is available at poc-slt-abc-completion-initialization-model

    • from scratch

      • you can use the code from train_no_empty_masking_custom_abc_config and remove the initialization from pre-trained model provided and train from scratch.

Dataset Generation

To generate SDF from different mesh datasets in form of LMDBs, please use the source from project page at sdf-generation-pipeline.

Citation

If you use this work, please cite it as:

@article{Zakeri2025POC,
  author  = {Zakeri, Faezeh and Braun, Raphael and Ruppert, Lukas and Lensch, Hendrik P.A.},
  title   = {POC-SLT: Partial Object Completion with SDF Latent Transformers},
  journal = {Proceedings of the Conference on Robots and Vision},
  year    = {2025},
  month   = {May 27},
  note    = {https://crv.pubpub.org/pub/yanc7d1w}
}
You can also find citation metadata files or by clicking **Cite this repository** on the right.