Spaces:
Running
Running
Update utils.py
Browse files
utils.py
CHANGED
@@ -57,9 +57,9 @@ SUBMIT_INTRODUCTION = """# Submit on MMEB Leaderboard Introduction
|
|
57 |
|
58 |
## ⚠ Please note that you need to submit the JSON file with the following format:
|
59 |
|
60 |
-
### ***Important Notes: We have released MMEB-V2 and will deprecate MMEB-V1 soon
|
|
|
61 |
### ***In V2, the detailed scores of each dataset will be included, and our code will automatically generate the results and calculate the overall scores. See the [**GitHub page**](https://github.com/TIGER-AI-Lab/VLM2Vec) for more information.***
|
62 |
-
### **A V2 Submission would look like this:**
|
63 |
```json
|
64 |
{
|
65 |
"metadata": {
|
@@ -103,7 +103,7 @@ SUBMIT_INTRODUCTION = """# Submit on MMEB Leaderboard Introduction
|
|
103 |
}
|
104 |
}
|
105 |
```
|
106 |
-
Note: We still accept the old format until 2025-06-30
|
107 |
Please refer to the [**GitHub page**](https://github.com/TIGER-AI-Lab/VLM2Vec) for detailed instructions about evaluating your model. \n
|
108 |
To submit, create a pull request and upload the generated JSON file to the ***scores*** folder, then send us an email at [email protected], including your model's information. \n We will review your submission and update the leaderboard accordingly. \n
|
109 |
Please also share any feedback or suggestions you have for improving the leaderboard experience. We appreciate your contributions to the MMEB community!
|
@@ -179,7 +179,7 @@ def process_model_size(size):
|
|
179 |
return 'unknown'
|
180 |
try:
|
181 |
val = float(size)
|
182 |
-
return val
|
183 |
except (ValueError, TypeError):
|
184 |
return 'unknown'
|
185 |
|
|
|
57 |
|
58 |
## ⚠ Please note that you need to submit the JSON file with the following format:
|
59 |
|
60 |
+
### ***Important Notes: We have released MMEB-V2 and will deprecate MMEB-V1 soon.*** \n
|
61 |
+
### ***All further submissions should be made using the V2 format (see following).*** \n
|
62 |
### ***In V2, the detailed scores of each dataset will be included, and our code will automatically generate the results and calculate the overall scores. See the [**GitHub page**](https://github.com/TIGER-AI-Lab/VLM2Vec) for more information.***
|
|
|
63 |
```json
|
64 |
{
|
65 |
"metadata": {
|
|
|
103 |
}
|
104 |
}
|
105 |
```
|
106 |
+
Note: We still accept the old format until 2025-06-30.
|
107 |
Please refer to the [**GitHub page**](https://github.com/TIGER-AI-Lab/VLM2Vec) for detailed instructions about evaluating your model. \n
|
108 |
To submit, create a pull request and upload the generated JSON file to the ***scores*** folder, then send us an email at [email protected], including your model's information. \n We will review your submission and update the leaderboard accordingly. \n
|
109 |
Please also share any feedback or suggestions you have for improving the leaderboard experience. We appreciate your contributions to the MMEB community!
|
|
|
179 |
return 'unknown'
|
180 |
try:
|
181 |
val = float(size)
|
182 |
+
return round(val, 3)
|
183 |
except (ValueError, TypeError):
|
184 |
return 'unknown'
|
185 |
|