| ## Getting Started | |
| ### Installation | |
| **1. Prepare the code and the environment** | |
| Git clone our repository, creating a python environment and ativate it via the following command | |
| ```bash | |
| git clone https://github.com/DLYuanGod/ArtGPT-4.git | |
| cd ArtGPT-4 | |
| conda env create -f environment.yml | |
| conda activate artgpt4 | |
| ``` | |
| **2. Prepare the pretrained Vicuna weights** | |
| The current version of MiniGPT-4 is built on the v0 versoin of Vicuna-13B. | |
| Please refer to our instruction [here](PrepareVicuna.md) | |
| to prepare the Vicuna weights. | |
| The final weights would be in a single folder in a structure similar to the following: | |
| ``` | |
| vicuna_weights | |
| ├── config.json | |
| ├── generation_config.json | |
| ├── pytorch_model.bin.index.json | |
| ├── pytorch_model-00001-of-00003.bin | |
| ... | |
| ``` | |
| Then, set the path to the vicuna weight in the model config file | |
| [here](minigpt4/configs/models/minigpt4.yaml#L16) at Line 16. | |
| **3. Prepare the pretrained MiniGPT-4 checkpoint** | |
| [Downlad](https://drive.google.com/file/d/1a4zLvaiDBr-36pasffmgpvH5P7CKmpze/view?usp=share_link) | |
| Then, set the path to the pretrained checkpoint in the evaluation config file | |
| in [eval_configs/minigpt4_eval.yaml](eval_configs/minigpt4_eval.yaml#L10) at Line 11. | |
| ### Launching Demo Locally | |
| Try out our demo [demo.py](demo.py) on your local machine by running | |
| ``` | |
| python demo.py --cfg-path eval_configs/minigpt4_eval.yaml --gpu-id 0 | |
| ``` | |
| ### Training | |
| The training of ArtGPT-4 contains two alignment stages. The training process for the step is consistent with that of [MiniGPT-4](https://minigpt-4.github.io/). | |
| **Datasets** | |
| We use [Laion-aesthetic](https://github.com/LAION-AI/laion-datasets/blob/main/laion-aesthetic.md) from the LAION-5B dataset, which amounts to approximately 200GB for the first 302 tar files. | |
| ## Acknowledgement | |
| + [MiniGPT-4](https://minigpt-4.github.io/) Our work is based on improvements to the model. | |
| ## License | |
| This repository is under [BSD 3-Clause License](LICENSE.md). | |
| Many codes are based on [Lavis](https://github.com/salesforce/LAVIS) with | |
| BSD 3-Clause License [here](LICENSE_Lavis.md). | |