Public version of our 3b-model trained to perform 9 specific task in english and french with high accuracy. The benchmark bellow is done with our evaluation pipeline.
Model Fine-tune from Qwen/Qwen2.5-3B-Instruct with a specific LoRA adapter.
The prompt of our 9 specific task might follow in a next post. The goal of our team is to specialized small LLM on our clients specific needs while keeping the highest quality on general tasks.
Our 3b models compete with GPT-mini on those specific tasks.
+----------------------+---------+------------+------------+
| qwen3b | Overall | Team score | loads fail |
+----------------------+---------+------------+------------+
| answer_reformulation | 0.73 | 0.74 | 0.99 |
| query_reformulation | 0.83 | 0.99 | 0 |
| summarization | 0.92 | --- | 0 |
| keyword_extraction | 0.78 | --- | 0 |
| fill_in_generation | 0.86 | --- | 0 |
| keyword_update | 0.66 | 0.89 | 2 |
| gqa | 0.81 | 0.63 | 3 |
| true_false | 0.77 | --- | 0 |
| mcq | 0.86 | --- | 0 |
| Total | 0.8 | --- | 5.99 |
+----------------------+---------+------------+------------+
- Downloads last month
- 1,425
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.