Update README.md
Browse files
README.md
CHANGED
|
@@ -4,4 +4,34 @@ language:
|
|
| 4 |
- en
|
| 5 |
base_model:
|
| 6 |
- mistralai/Mistral-7B-Instruct-v0.3
|
| 7 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
- en
|
| 5 |
base_model:
|
| 6 |
- mistralai/Mistral-7B-Instruct-v0.3
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
## Model Information
|
| 10 |
+
|
| 11 |
+
We release the full ranking model RankMistral100 distilled from GPT-4o-2024-08-06 used in **Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models**.
|
| 12 |
+
|
| 13 |
+
<p align="left">
|
| 14 |
+
Useful links: 📝 <a href="" target="_blank">Paper</a> • 🤗 <a href="https://huggingface.co/datasets/liuwenhan/msmarco_full_ranking_list" target="_blank">Dataset</a> • </a> 🧩 <a href="https://github.com/8421BCD/fullrank" target="_blank">Github</a>
|
| 15 |
+
</p>
|
| 16 |
+
|
| 17 |
+
## Training framework
|
| 18 |
+
Our full ranking model aims to directly rerank 100 passages at a time, abandoning the sliding window strategy. We propose a multi-pass sliding window approach for generating the full ranking list as label and design a importance-aware training loss for optimization.
|
| 19 |
+
<img src="https://8421bcd.oss-cn-beijing.aliyuncs.com/img/image-20241218200920116.png" alt="image-20241218200920116" style="zoom: 45%;" />
|
| 20 |
+
|
| 21 |
+
## Backbone Model
|
| 22 |
+
RankMistral100 is finetuned from https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3.
|
| 23 |
+
|
| 24 |
+
## Performance
|
| 25 |
+
Our model surpuss the strong baseline RankZephyr with 1.2 points on BEIR Avg.
|
| 26 |
+
| Models | Covid | DBPedia | SciFact | NFCorpus | Signal | Robust04 | Touche | News | Avg. |
|
| 27 |
+
| ------------------------- | ----- | ------- | ------- | -------- | ------ | -------- | ------ | ----- | --------- |
|
| 28 |
+
| BM25 | 59.47 | 31.80 | 67.89 | 33.75 | 33.04 | 40.70 | 44.22 | 39.52 | 43.80 |
|
| 29 |
+
| monoBERT (340M) | 73.45 | 41.69 | 62.22 | 34.92 | 30.63 | 44.21 | 30.26 | 47.03 | 45.55 |
|
| 30 |
+
| monoT5 (220M) | 75.94 | 42.43 | 65.07 | 35.42 | 31.20 | 44.15 | 30.35 | 46.98 | 46.44 |
|
| 31 |
+
| RankVicuna (7B) | 79.19 | 44.51 | 70.67 | 34.51 | 34.24 | 48.33 | 33.00 | 47.15 | 48.95 |
|
| 32 |
+
| RankZepeyer (7B) | 82.92 | 44.42 | 75.42 | 38.26 | 31.41 | 53.73 | 30.22 | 52.80 | 51.15 |
|
| 33 |
+
| RankMistral<sub>100</sub> (7B) | 82.24 | 43.54 | 77.04 | 39.14 | 33.99 | 57.91 | 34.63 | 50.59 | **52.40** |
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
🌹 If you use this model, please ✨star our GitHub repository to support us. Your star means a lot!
|
| 37 |
+
|