Update README.md
Browse files
README.md
CHANGED
@@ -48,7 +48,7 @@ print(tokenizer.decode(output_ids[input_ids.shape[1]:], skip_special_tokens=True
|
|
48 |
```
|
49 |
|
50 |
## Model evaluation
|
51 |
-
We conduct evaluation on 9 commonly-used benchmarks, including 5 academic VQA benchmarks and 4 popular MLLM benchmarks, to compare our Imp model with LLaVA (7B) and existing MSLMs
|
52 |
|
53 |
| Models | Size | VQAv2 | GQA |VizWiz | SQA (IMG) | TextVQA | POPE | MME | MMB |MM-Vet|
|
54 |
|:--------:|:-----:|:----:|:----:|:-------------:|:--------:|:-----:|:----:|:-------:|:-------:|:-------:|
|
@@ -56,8 +56,8 @@ We conduct evaluation on 9 commonly-used benchmarks, including 5 academic VQA be
|
|
56 |
| [TinyGPT-V](https://huggingface.co/Tyrannosaurus/TinyGPT-V) | 3B | - | 33.60 | 24.80 | - | - | -| - | - |-|
|
57 |
| [LLaVA-Phi](https://github.com/zhuyiche/llava-phi) | 3B | 71.40 | - | 35.90 | 68.40 | 48.60 | 85.00 | 1335.1 | 59.80 |28.9|
|
58 |
| [MobileVLM](https://huggingface.co/mtgv/MobileVLM-3B) | 3B | - | 59.00 | - | 61.00 | 47.50 | 84.90 | 1288.9 | 59.60 |-|
|
59 |
-
| [MC-LLaVA-3b](https://huggingface.co/visheratin/MC-LLaVA-3b) | 3B | 64.24 | 49.
|
60 |
-
| **Imp-v0 (ours)** | 3B | **79.45** | 58.55 | **50.09** |**69.96**| **59.38** | **88.02**| 1434 | **66.49** |**33.1**|
|
61 |
|
62 |
### Example
|
63 |
![example1](images/example1.png)
|
|
|
48 |
```
|
49 |
|
50 |
## Model evaluation
|
51 |
+
We conduct evaluation on 9 commonly-used benchmarks, including 5 academic VQA benchmarks and 4 popular MLLM benchmarks, to compare our Imp model with LLaVA (7B) and existing MSLMs of similar model sizes.
|
52 |
|
53 |
| Models | Size | VQAv2 | GQA |VizWiz | SQA (IMG) | TextVQA | POPE | MME | MMB |MM-Vet|
|
54 |
|:--------:|:-----:|:----:|:----:|:-------------:|:--------:|:-----:|:----:|:-------:|:-------:|:-------:|
|
|
|
56 |
| [TinyGPT-V](https://huggingface.co/Tyrannosaurus/TinyGPT-V) | 3B | - | 33.60 | 24.80 | - | - | -| - | - |-|
|
57 |
| [LLaVA-Phi](https://github.com/zhuyiche/llava-phi) | 3B | 71.40 | - | 35.90 | 68.40 | 48.60 | 85.00 | 1335.1 | 59.80 |28.9|
|
58 |
| [MobileVLM](https://huggingface.co/mtgv/MobileVLM-3B) | 3B | - | 59.00 | - | 61.00 | 47.50 | 84.90 | 1288.9 | 59.60 |-|
|
59 |
+
| [MC-LLaVA-3b](https://huggingface.co/visheratin/MC-LLaVA-3b) | 3B | 64.24 | 49.60 | 24.88 | - | 38.59 | 80.59 | - | - |-|
|
60 |
+
| **Imp-v0 (ours)** | 3B | **79.45** | 58.55 | **50.09** |**69.96**| **59.38** | **88.02**| 1434.0 | **66.49** |**33.1**|
|
61 |
|
62 |
### Example
|
63 |
![example1](images/example1.png)
|