Spaces:
Running
on
Zero
Splatt3R: Zero-shot Gaussian Splatting from Uncalibarated Image Pairs
Official implementation of Zero-shot Gaussian Splatting from Uncalibarated Image Pairs
Links removed for anonymity:
Project Page, Splatt3R arXiv
Installation
- Clone Splatt3R
git clone <redacted github link>
cd splatt3r
- Setup Anaconda Environment
conda env create -f environment.yml
pip install git+https://github.com/dcharatan/diff-gaussian-rasterization-modified
- (Optional) Compile the CUDA kernels for RoPE (as in MASt3R and CroCo v2)
cd src/dust3r_src/croco/models/curope/
python setup.py build_ext --inplace
cd ../../../../../
Checkpoints
We train our model using the pretrained MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric
checkpoint from the MASt3R authors, available from the MASt3R GitHub repo. This checkpoint is placed at the file path checkpoints/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth
.
A pretrained Splatt3R model can be downloaded here (redacted link).
Data
We use ScanNet++ to train our model. We download the data from the official ScanNet++ homepage and process the data using SplaTAM's modified version of the ScanNet++ toolkit. We save the processed data to the 'processed' subfolder of the ScanNet++ root directory.
Our generated test coverage files, and our training and testing splits, can be downloaded here (redacted link), and placed in data/scannetpp
.
Demo
The Gradio demo can be run using python demo.py <checkpoint_path>
, replacing <checkpoint_path>
with the trained network path. A checkpoint will be available for the public release of this code.
This demo generates a .ply
file that represents the scene, which can be downloaded and rendered using online 3D Gaussian Splatting viewers such as here or here.
Training
Our training run can be recreated by running python main.py configs/main.yaml
. Other configurations can be found in the configs
folder.
BibTeX
Forthcoming arXiv citation