MINGYISU commited on
Commit
aa65c27
·
1 Parent(s): 4b853a0
Files changed (2) hide show
  1. results.csv +2 -1
  2. utils.py +4 -0
results.csv CHANGED
@@ -23,4 +23,5 @@ VLM2Vec (Qwen2-VL-7B-LoRA-HighRes),8.29,TIGER-Lab,65.8,62.6,57.8,69.9,81.7
23
  VLM2Vec (Qwen2-VL-2B-LoRA-HighRes),2.21,TIGER-Lab,59.3,59.0,49.4,65.4,73.4
24
  LLaVE-7B,8.03,Self-Reported,70.3,65.7,65.4,70.9,91.9
25
  LLaVE-2B,1.95,Self-Reported,65.2,62.1,60.2,65.2,84.9
26
- LLaVE-0.5B,0.894,Self-Reported,59.1,57.4,50.3,59.8,82.9
 
 
23
  VLM2Vec (Qwen2-VL-2B-LoRA-HighRes),2.21,TIGER-Lab,59.3,59.0,49.4,65.4,73.4
24
  LLaVE-7B,8.03,Self-Reported,70.3,65.7,65.4,70.9,91.9
25
  LLaVE-2B,1.95,Self-Reported,65.2,62.1,60.2,65.2,84.9
26
+ LLaVE-0.5B,0.894,Self-Reported,59.1,57.4,50.3,59.8,82.9
27
+ QQMM,8.297,Self-Reported,72.175,70.07,69.52,71.175,87.075
utils.py CHANGED
@@ -41,6 +41,8 @@ or a combination of both. MMEB is divided into 20 in-distribution datasets, whic
41
  training, and 16 out-of-distribution datasets, reserved for evaluation.
42
 
43
  The detailed explanation of the benchmark and datasets can be found in our paper: https://doi.org/10.48550/arXiv.2410.05160.
 
 
44
  """
45
 
46
  TABLE_INTRODUCTION = """"""
@@ -94,6 +96,8 @@ SUBMIT_INTRODUCTION = """# Submit on MMEB Leaderboard Introduction
94
  },
95
  ]
96
  ```
 
 
97
  Please send us an email at [email protected], attaching the JSON file. We will review your submission and update the leaderboard accordingly.
98
  """
99
 
 
41
  training, and 16 out-of-distribution datasets, reserved for evaluation.
42
 
43
  The detailed explanation of the benchmark and datasets can be found in our paper: https://doi.org/10.48550/arXiv.2410.05160.
44
+ Github link: https://github.com/TIGER-AI-Lab/VLM2Vec
45
+ Overview: https://tiger-ai-lab.github.io/VLM2Vec/
46
  """
47
 
48
  TABLE_INTRODUCTION = """"""
 
96
  },
97
  ]
98
  ```
99
+ You may refer to the Github page for instructions about evaluating your model.
100
+ Github link: https://github.com/TIGER-AI-Lab/VLM2Vec
101
  Please send us an email at [email protected], attaching the JSON file. We will review your submission and update the leaderboard accordingly.
102
  """
103