Stable-X commited on
Commit
91786b4
1 Parent(s): 1164fc6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -123
README.md CHANGED
@@ -1,128 +1,6 @@
1
  ---
2
- title: StableSpann3R
3
  app_file: app.py
4
  sdk: gradio
5
  sdk_version: 4.42.0
6
  ---
7
- # 3D Reconstruction with Spatial Memory
8
-
9
- ### [Paper](https://arxiv.org/abs/2408.16061) | [Project Page](https://hengyiwang.github.io/projects/spanner) | [Video](https://hengyiwang.github.io/projects/spanner/videos/spanner_intro.mp4)
10
-
11
- > 3D Reconstruction with Spatial Memory <br />
12
- > [Hengyi Wang](https://hengyiwang.github.io/), [Lourdes Agapito](http://www0.cs.ucl.ac.uk/staff/L.Agapito/)<br />
13
- > arXiv 2024
14
-
15
- <p align="center">
16
- <a href="">
17
- <img src="./assets/spann3r_teaser_white.gif" alt="Logo" width="90%">
18
- </a>
19
- </p>
20
-
21
- ## Installation
22
-
23
- 1. Clone Spann3R
24
-
25
- ```
26
- git clone https://github.com/HengyiWang/spann3r.git
27
- cd spann3r
28
- ```
29
-
30
- 2. Create conda environment
31
-
32
- ```
33
- conda create -n spann3r python=3.9 cmake=3.14.0
34
- conda install pytorch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 pytorch-cuda=11.8 -c pytorch -c nvidia # use the correct version of cuda for your system
35
-
36
- pip install -r requirements.txt
37
-
38
- # Open3D has a bug from 0.16.0, please use dev version
39
- pip install -U -f https://www.open3d.org/docs/latest/getting_started.html open3d
40
- ```
41
-
42
- 3. Compile cuda kernels for RoPE
43
-
44
- ```
45
- cd croco/models/curope/
46
- python setup.py build_ext --inplace
47
- cd ../../../
48
- ```
49
-
50
- 4. Download the DUSt3R checkpoint
51
-
52
- ```
53
- mkdir checkpoints
54
- cd checkpoints
55
- # Download DUSt3R checkpoints
56
- wget https://download.europe.naverlabs.com/ComputerVision/DUSt3R/DUSt3R_ViTLarge_BaseDecoder_512_dpt.pth
57
- ```
58
-
59
- 5. Download our [checkpoint](https://drive.google.com/drive/folders/1bqtcVf8lK4VC8LgG-SIGRBECcrFqM7Wy?usp=sharing) and place it under `./checkpoints`
60
-
61
- ## Demo
62
-
63
- 1. Download the [example data](https://drive.google.com/drive/folders/1bqtcVf8lK4VC8LgG-SIGRBECcrFqM7Wy?usp=sharing) (2 scenes from [map-free-reloc](https://github.com/nianticlabs/map-free-reloc)) and unzip it as `./examples`
64
-
65
- 2. Run demo:
66
-
67
- ```
68
- python demo.py --demo_path ./examples/s00567 --kf_every 10 --vis
69
- ```
70
-
71
- For visualization `--vis`, it will give you a window to adjust the rendering view. Once you find the view to render, please click `space key` and close the window. The code will then do the rendering of the incremental reconstruction.
72
-
73
-
74
-
75
- ## Training and Evaluation
76
-
77
- ### Datasets
78
-
79
- We use Habitat, ScanNet++, ScanNet, ArkitScenes, Co3D, and BlendedMVS to train our model. Please refer to [data_preprocess.md](docs/data_preprocess.md).
80
-
81
- ### Train
82
-
83
- Please use the following command to train our model:
84
-
85
- ```
86
- torchrun --nproc_per_node 8 train.py --batch_size 4
87
- ```
88
-
89
- ### Eval
90
-
91
- Please use the following command to evaluate our model:
92
-
93
- ```
94
- python eval.py
95
- ```
96
-
97
-
98
-
99
-
100
- ## Acknowledgement
101
-
102
- Our code, data preprocessing pipeline, and evaluation scripts are based on several awesome repositories:
103
-
104
- - [DUSt3R](https://github.com/naver/dust3r)
105
- - [SplaTAM](https://github.com/spla-tam/SplaTAM)
106
- - [NeRFStudio](https://github.com/nerfstudio-project/nerfstudio)
107
- - [MVSNet](https://github.com/YoYo000/MVSNet)
108
- - [NICE-SLAM](https://github.com/cvg/nice-slam)
109
- - [NeuralRGBD](https://github.com/dazinovic/neural-rgbd-surface-reconstruction)
110
- - [SimpleRecon](https://github.com/nianticlabs/simplerecon)
111
-
112
- We thank the authors for releasing their code!
113
-
114
- The research presented here has been supported by a sponsored research award from Cisco Research and the UCL Centre for Doctoral Training in Foundational AI under UKRI grant number EP/S021566/1. This project made use of time on Tier 2 HPC facility JADE2, funded by EPSRC (EP/T022205/1).
115
-
116
- ## Citation
117
-
118
- If you find our code or paper useful for your research, please consider citing:
119
-
120
- ```
121
- @article{wang20243d,
122
- title={3D Reconstruction with Spatial Memory},
123
- author={Wang, Hengyi and Agapito, Lourdes},
124
- journal={arXiv preprint arXiv:2408.16061},
125
- year={2024}
126
- }
127
- ```
128
-
 
1
  ---
2
+ title: StableRecon
3
  app_file: app.py
4
  sdk: gradio
5
  sdk_version: 4.42.0
6
  ---