create README.md

README.md

@@ -0,0 +1,60 @@
+---
+license: apache-2.0
+metrics:
+- accuracy
+base_model:
+- liuhaotian/llava-v1.5-7b
+---
+# LLaVA-3D
+
+##  Table of Contents
+
+1. [Model Summary](##model-summary)
+2. [Use](##use)
+3. [Limitations](##limitations)
+4. [Training](##training)
+5. [License](##license)
+6. [Citation](##citation)
+
+## Model Summary
+
+The LLaVA-3D model is a 7B parameter models trained on LLaVA-3D-Instruct-1M, based on LLaVA-v1.5-7B.
+
+- **Repository:** [ZCMax/LLaVA-3D](https://github.com/ZCMax/LLaVA-3D)
+- **Project Website:** [zcmax.github.io/projects/LLaVA-3D](https://zcmax.github.io/projects/LLaVA-3D/)
+- **Paper:** [LLaVA-3D](https://arxiv.org/abs/2409.18125)
+- **Point of Contact:** [Chenming Zhu](mailto:[email protected])
+- **Languages:** English
+
+
+## Use
+
+### Intended use
+
+The model was trained on LLaVA-3D-Instruct-1M and has the ability to interact with the single image for 2D tasks and posed RBG-D images for 3D tasks. 
+
+**Feel free to share your generations in the Community tab!**
+
+# Training
+
+## Model
+
+- **Pretraining Stage:** scene-level and region-level caption data, 1 epoch, projector
+- **Instructing Tuning Stage:** A mixture of 1M high-quality 2D and 3D data, 1 epoch, full model
+- **Precision:** bfloat16
+
+## Hardware & Software
+
+- **GPUs:** 8 * Nvidia Tesla A100 (for whole model series training)
+- **Orchestration:** [Huggingface Trainer](https://huggingface.co/docs/transformers/main_classes/trainer)
+- **Neural networks:** [PyTorch](https://github.com/pytorch/pytorch)
+
+# Citation
+```
+@article{zhu2024llava,
+  title={LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness},
+  author={Zhu, Chenming and Wang, Tai and Zhang, Wenwei and Pang, Jiangmiao and Liu, Xihui},
+  journal={arXiv preprint arXiv:2409.18125},
+  year={2024}
+}
+```