Update README.md
Browse files
README.md
CHANGED
@@ -5,13 +5,13 @@ datasets:
|
|
5 |
license: cc-by-nc-4.0
|
6 |
pipeline_tag: sentence-similarity
|
7 |
---
|
8 |
-
These are LoRA adaption weights for [mT5](https://huggingface.co/google/mt5-xxl) encoder.
|
9 |
|
10 |
-
## Multilingual Sentence T5
|
11 |
This model is a multilingual extension of Sentence T5 and was created using the [mT5](https://huggingface.co/google/mt5-xxl) encoder. It is proposed in this [paper](https://arxiv.org/abs/2403.17528).
|
12 |
-
|
13 |
|
14 |
-
###
|
15 |
The model was trained on the XNLI dataset.
|
16 |
|
17 |
### Framework versions
|
@@ -19,7 +19,7 @@ The model was trained on the XNLI dataset.
|
|
19 |
|
20 |
- PEFT 0.4.0.dev0
|
21 |
|
22 |
-
##
|
23 |
0. If you have not installed peft, please do so.
|
24 |
```
|
25 |
pip install -q git+https://github.com/huggingface/transformers.git@main git+https://github.com/huggingface/peft.git
|
@@ -34,7 +34,7 @@ model.enable_input_require_grads()
|
|
34 |
model.gradient_checkpointing_enable()
|
35 |
model: PeftModel = PeftModel.from_pretrained(model, "pkshatech/m-ST5")
|
36 |
```
|
37 |
-
2. To obtain sentence embedding, use
|
38 |
```
|
39 |
tokenizer = AutoTokenizer.from_pretrained("google/mt5-xxl", use_fast=False)
|
40 |
model.eval()
|
|
|
5 |
license: cc-by-nc-4.0
|
6 |
pipeline_tag: sentence-similarity
|
7 |
---
|
8 |
+
These are LoRA adaption weights for the [mT5](https://huggingface.co/google/mt5-xxl) encoder.
|
9 |
|
10 |
+
## Multilingual Sentence T5 (m-ST5)
|
11 |
This model is a multilingual extension of Sentence T5 and was created using the [mT5](https://huggingface.co/google/mt5-xxl) encoder. It is proposed in this [paper](https://arxiv.org/abs/2403.17528).
|
12 |
+
m-ST5 is an encoder for sentence embedding, and its performance has been verified in cross-lingual semantic textual similarity (STS) and sentence retrieval tasks.
|
13 |
|
14 |
+
### Training Data
|
15 |
The model was trained on the XNLI dataset.
|
16 |
|
17 |
### Framework versions
|
|
|
19 |
|
20 |
- PEFT 0.4.0.dev0
|
21 |
|
22 |
+
## How to use
|
23 |
0. If you have not installed peft, please do so.
|
24 |
```
|
25 |
pip install -q git+https://github.com/huggingface/transformers.git@main git+https://github.com/huggingface/peft.git
|
|
|
34 |
model.gradient_checkpointing_enable()
|
35 |
model: PeftModel = PeftModel.from_pretrained(model, "pkshatech/m-ST5")
|
36 |
```
|
37 |
+
2. To obtain sentence embedding, use mean pooling.
|
38 |
```
|
39 |
tokenizer = AutoTokenizer.from_pretrained("google/mt5-xxl", use_fast=False)
|
40 |
model.eval()
|