Update pipeline tag to `text-ranking` and fix quickstart code
Browse filesThis PR updates the model card to improve its accuracy and usability:
1. **Metadata Update:** The `pipeline_tag` has been changed from `text-generation` to `text-ranking`. This more precisely reflects the model's primary function as a judge model designed for evaluating and comparing (ranking) responses from other language models. This update will enhance the model's discoverability on the Hugging Face Hub, allowing users to easily find it when filtering for `text-ranking` capabilities (e.g., via `https://huggingface.co/models?pipeline_tag=text-ranking`).
2. **Quickstart Code Fix:** A minor bug in the Python quickstart example has been resolved. The variables `model_name` were replaced with `model_path` in the `AutoModelForCausalLM.from_pretrained` and `AutoTokenizer.from_pretrained` calls. This ensures the sample code is directly runnable and correctly loads the model, preventing a `NameError` for users trying to replicate the example.
These changes contribute to a more accurate and user-friendly model card.
@@ -1,10 +1,10 @@
|
|
1 |
---
|
2 |
-
license: apache-2.0
|
3 |
-
pipeline_tag: text-generation
|
4 |
library_name: transformers
|
|
|
|
|
5 |
tags:
|
6 |
-
|
7 |
-
|
8 |
---
|
9 |
|
10 |
# CompassJudger-2
|
@@ -59,11 +59,11 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
59 |
model_path = "opencompass/CompassJudger-2-7B-Instruct"
|
60 |
|
61 |
model = AutoModelForCausalLM.from_pretrained(
|
62 |
-
|
63 |
torch_dtype="auto",
|
64 |
device_map="auto"
|
65 |
)
|
66 |
-
tokenizer = AutoTokenizer.from_pretrained(
|
67 |
|
68 |
# Example: Pair-wise Comparison
|
69 |
prompt = """
|
@@ -125,7 +125,7 @@ CompassJudger-2 sets a new state-of-the-art for judge models, outperforming gene
|
|
125 |
| Skywork-Critic-Llama-3.1-70B | 52.41 | 50.65 | 65.50 | 93.30 | 65.47 |
|
126 |
| RISE-Judge-Qwen2.5-32B | 56.42 | 63.87 | 73.70 | 92.70 | 71.67 |
|
127 |
| **CompassJudger-2-32B-Instruct** | **62.21** | **65.48** | 72.98 | **92.62** | **73.32** |
|
128 |
-
| **General Models (for reference)** | | | | |
|
129 |
| Qwen2.5-32B-Instruct | 62.97 | 59.84 | 74.99 | 85.61 | 70.85 |
|
130 |
| DeepSeek-V3-0324 | 64.43 | 59.68 | 78.16 | 85.17 | 71.86 |
|
131 |
| Qwen3-235B-A22B | 61.40 | 65.97 | 75.59 | 84.68 | 71.91 |
|
|
|
1 |
---
|
|
|
|
|
2 |
library_name: transformers
|
3 |
+
license: apache-2.0
|
4 |
+
pipeline_tag: text-ranking
|
5 |
tags:
|
6 |
+
- llm-as-judge
|
7 |
+
- qwen2
|
8 |
---
|
9 |
|
10 |
# CompassJudger-2
|
|
|
59 |
model_path = "opencompass/CompassJudger-2-7B-Instruct"
|
60 |
|
61 |
model = AutoModelForCausalLM.from_pretrained(
|
62 |
+
model_path,
|
63 |
torch_dtype="auto",
|
64 |
device_map="auto"
|
65 |
)
|
66 |
+
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
67 |
|
68 |
# Example: Pair-wise Comparison
|
69 |
prompt = """
|
|
|
125 |
| Skywork-Critic-Llama-3.1-70B | 52.41 | 50.65 | 65.50 | 93.30 | 65.47 |
|
126 |
| RISE-Judge-Qwen2.5-32B | 56.42 | 63.87 | 73.70 | 92.70 | 71.67 |
|
127 |
| **CompassJudger-2-32B-Instruct** | **62.21** | **65.48** | 72.98 | **92.62** | **73.32** |
|
128 |
+
| **General Models (for reference)** | | | | | |\
|
129 |
| Qwen2.5-32B-Instruct | 62.97 | 59.84 | 74.99 | 85.61 | 70.85 |
|
130 |
| DeepSeek-V3-0324 | 64.43 | 59.68 | 78.16 | 85.17 | 71.86 |
|
131 |
| Qwen3-235B-A22B | 61.40 | 65.97 | 75.59 | 84.68 | 71.91 |
|