Update README.md
Browse files
README.md
CHANGED
@@ -2,10 +2,12 @@
|
|
2 |
license: apache-2.0
|
3 |
pipeline_tag: visual-question-answering
|
4 |
---
|
5 |
-
# 😈Imp
|
6 |
|
7 |
\[Technical report (coming soon)\] [[Demo](https://xmbot,net/imp/)\] [[Github](https://github.com/MILVLG/imp)\]
|
8 |
|
|
|
|
|
9 |
The Imp project aims to provide a family of a strong multimodal `small` language models (MSLMs). Our `imp-v1-3b` is a strong MSLM with only **3B** parameters, which is build upon a small yet powerful SLM [Phi-2 ](https://huggingface.co/microsoft/phi-2)(2.7B) and a powerful visual encoder [SigLIP ](https://huggingface.co/google/siglip-so400m-patch14-384)(0.4B), and trained on the [LLaVA-v1.5](https://github.com/haotian-liu/LLaVA) training set.
|
10 |
|
11 |
As shown in the Table below, `imp-v1-3b` significantly outperforms the counterparts of similar model sizes, and even achieves slightly better performance than the strong LLaVA-7B model on various multimodal benchmarks.
|
@@ -15,7 +17,7 @@ We release our model weights and provide an example below to run our model . Det
|
|
15 |
|
16 |
## How to use
|
17 |
|
18 |
-
You can use the following code for model inference. We minimize the required dependency libraries that only the `transformers` and `torch` packages
|
19 |
|
20 |
```Python
|
21 |
import torch
|
|
|
2 |
license: apache-2.0
|
3 |
pipeline_tag: visual-question-answering
|
4 |
---
|
5 |
+
# 😈 Imp
|
6 |
|
7 |
\[Technical report (coming soon)\] [[Demo](https://xmbot,net/imp/)\] [[Github](https://github.com/MILVLG/imp)\]
|
8 |
|
9 |
+
## Introduction
|
10 |
+
|
11 |
The Imp project aims to provide a family of a strong multimodal `small` language models (MSLMs). Our `imp-v1-3b` is a strong MSLM with only **3B** parameters, which is build upon a small yet powerful SLM [Phi-2 ](https://huggingface.co/microsoft/phi-2)(2.7B) and a powerful visual encoder [SigLIP ](https://huggingface.co/google/siglip-so400m-patch14-384)(0.4B), and trained on the [LLaVA-v1.5](https://github.com/haotian-liu/LLaVA) training set.
|
12 |
|
13 |
As shown in the Table below, `imp-v1-3b` significantly outperforms the counterparts of similar model sizes, and even achieves slightly better performance than the strong LLaVA-7B model on various multimodal benchmarks.
|
|
|
17 |
|
18 |
## How to use
|
19 |
|
20 |
+
You can use the following code for model inference. We minimize the required dependency libraries that only the `transformers` and `torch` packages need to be installed. The latest version of `transformers` is ok but we recommand v4.31.0. The format of text instruction is similar to [LLaVA](https://github.com/haotian-liu/LLaVA).
|
21 |
|
22 |
```Python
|
23 |
import torch
|