Spaces:
Running
Running
MINGYISU
commited on
Commit
·
aa65c27
1
Parent(s):
4b853a0
add qqmm
Browse files- results.csv +2 -1
- utils.py +4 -0
results.csv
CHANGED
@@ -23,4 +23,5 @@ VLM2Vec (Qwen2-VL-7B-LoRA-HighRes),8.29,TIGER-Lab,65.8,62.6,57.8,69.9,81.7
|
|
23 |
VLM2Vec (Qwen2-VL-2B-LoRA-HighRes),2.21,TIGER-Lab,59.3,59.0,49.4,65.4,73.4
|
24 |
LLaVE-7B,8.03,Self-Reported,70.3,65.7,65.4,70.9,91.9
|
25 |
LLaVE-2B,1.95,Self-Reported,65.2,62.1,60.2,65.2,84.9
|
26 |
-
LLaVE-0.5B,0.894,Self-Reported,59.1,57.4,50.3,59.8,82.9
|
|
|
|
23 |
VLM2Vec (Qwen2-VL-2B-LoRA-HighRes),2.21,TIGER-Lab,59.3,59.0,49.4,65.4,73.4
|
24 |
LLaVE-7B,8.03,Self-Reported,70.3,65.7,65.4,70.9,91.9
|
25 |
LLaVE-2B,1.95,Self-Reported,65.2,62.1,60.2,65.2,84.9
|
26 |
+
LLaVE-0.5B,0.894,Self-Reported,59.1,57.4,50.3,59.8,82.9
|
27 |
+
QQMM,8.297,Self-Reported,72.175,70.07,69.52,71.175,87.075
|
utils.py
CHANGED
@@ -41,6 +41,8 @@ or a combination of both. MMEB is divided into 20 in-distribution datasets, whic
|
|
41 |
training, and 16 out-of-distribution datasets, reserved for evaluation.
|
42 |
|
43 |
The detailed explanation of the benchmark and datasets can be found in our paper: https://doi.org/10.48550/arXiv.2410.05160.
|
|
|
|
|
44 |
"""
|
45 |
|
46 |
TABLE_INTRODUCTION = """"""
|
@@ -94,6 +96,8 @@ SUBMIT_INTRODUCTION = """# Submit on MMEB Leaderboard Introduction
|
|
94 |
},
|
95 |
]
|
96 |
```
|
|
|
|
|
97 |
Please send us an email at [email protected], attaching the JSON file. We will review your submission and update the leaderboard accordingly.
|
98 |
"""
|
99 |
|
|
|
41 |
training, and 16 out-of-distribution datasets, reserved for evaluation.
|
42 |
|
43 |
The detailed explanation of the benchmark and datasets can be found in our paper: https://doi.org/10.48550/arXiv.2410.05160.
|
44 |
+
Github link: https://github.com/TIGER-AI-Lab/VLM2Vec
|
45 |
+
Overview: https://tiger-ai-lab.github.io/VLM2Vec/
|
46 |
"""
|
47 |
|
48 |
TABLE_INTRODUCTION = """"""
|
|
|
96 |
},
|
97 |
]
|
98 |
```
|
99 |
+
You may refer to the Github page for instructions about evaluating your model.
|
100 |
+
Github link: https://github.com/TIGER-AI-Lab/VLM2Vec
|
101 |
Please send us an email at [email protected], attaching the JSON file. We will review your submission and update the leaderboard accordingly.
|
102 |
"""
|
103 |
|