docs/data_preprocess.md

Datasets

Training datasets

ScanNet++

1. Download the dataset
2. Pre-process the dataset using pre-process code in SplaTAM to generate undistorted DSLR depth.
3. Place it in ./data/scannetpp

NOTE: Scannetpp is a great dataset for debugging and test purposes.

ScanNet

1. Download the dataset
2. Extract and organize the dataset using pre-process script in SimpleRecon
3. Place it in ./data/scannet

Habitat

1. Download the dataset
2. Render 5-frame video as in Croco. You may want to read the instructions
3. Place it in ./data/habitat_5frame

NOTE: We render the 5-frame using aminimum covisiblity of 0.1. This can improve the rendering speed, but the generated data may not be optimal for training Spann3R.

ArkitScenes

1. Download the dataset
2. Place it in ./data/arkit_lowres

NOTE: Due to the limit of storage, we use low-resolution input to supervise Spann3R. Ideally, you can use a higher resolution i.e. vga_wide, as in DUSt3R, for training.

Co3D

1. Download the dataset
2. Pre-process dataset as in DUSt3R
3. Place it in ./data/co3d_preprocessed_50

NOTE: For Co3D, we use two sampling strategies to train our model, one is the same as in DUSt3R, another is our own sampling strategy as in other datasets that contain videos.

BlendedMVS

1. Download the dataset
2. Place it in ./data/blendmvg

Evaluation datasets

7 Scenes

1. Download the dataset. You may want to use code in SimpleRecon to download the data
2. Use pre-process code in SimpleRecon to generate pseudo gt depth
3. Place it in ./data/7scenes

Neural RGBD

1. Download the dataset
2. Place it in ./data/neural_rgbd

DTU

1. Download the dataset. Note that we render the depth as in MVSNet and use our own mask annotations for evaluation. You can download our pre-processed DTU that contains the rendered depth map for evaluation here.
2. Place it in ./data/dtu_test