Thacio Garcia Scandaroli
commited on
Commit
•
fedb6f2
1
Parent(s):
54fbe35
Update README.md
Browse files
README.md
CHANGED
@@ -18,10 +18,10 @@ library_name: transformers
|
|
18 |
|
19 |
<!-- Provide a longer summary of what this model is. -->
|
20 |
|
21 |
-
ULT5-pt é um modelo de arquitetura T5-v1.1 treinado com o framework UL2 - [Unifying Language Learning Paradigms](https://arxiv.org/abs/2205.05131v1), que utiliza Mixture-of-Denoisers (MoD), o qual combina o objetivo de Causal Language Modeling (CLM) com Span Corruption.
|
22 |
-
|
23 |
*ULT5-pt is a T5-v1.1 architecture model trained using the UL2 - Unifying Language Learning Paradigms framework, which uses Mixture-of-Denoisers (MoD), combining Causal Language Modeling (CLM) objective with Span Corruption.*
|
24 |
|
|
|
|
|
25 |
| Model | Parameters |
|
26 |
| :-: | :-: |
|
27 |
| [thacio/ult5-pt-small](https://huggingface.co/thacio/ult5-pt-small) | 82.4M |
|
@@ -54,11 +54,11 @@ Os modos de *span corruption* podem ser acionados adicionado ao início do text
|
|
54 |
Os autores do UL2 apontam uma possivel diferença no resultado do fine-tune dependendo do modo ativado.
|
55 |
Porém, para o ult5-pt, não se notou diferença nos testes de benchmark.
|
56 |
|
57 |
-
*Fine-tunning is the recommended use for the model
|
58 |
|
59 |
-
A tutorial (in portuguse) in notebook format for decoder and encoder-decoder (T5) model fine-tuning was provided: [Fine-tune Large Language Models](link here)
|
60 |
|
61 |
-
Span corruption modes can be activated by adding the prefixes '<|NLU|>' and '<|NLG|>' to the beginning of the text. The UL2 authors point out a possible difference in the fine-tuning result depending on the activated mode. However, for ult5-pt, no difference was noticed in benchmark tests.*
|
62 |
|
63 |
### Direct Use
|
64 |
|
|
|
18 |
|
19 |
<!-- Provide a longer summary of what this model is. -->
|
20 |
|
|
|
|
|
21 |
*ULT5-pt is a T5-v1.1 architecture model trained using the UL2 - Unifying Language Learning Paradigms framework, which uses Mixture-of-Denoisers (MoD), combining Causal Language Modeling (CLM) objective with Span Corruption.*
|
22 |
|
23 |
+
ULT5-pt é um modelo de arquitetura T5-v1.1 treinado com o framework UL2 - [Unifying Language Learning Paradigms](https://arxiv.org/abs/2205.05131v1), que utiliza Mixture-of-Denoisers (MoD), o qual combina o objetivo de Causal Language Modeling (CLM) com Span Corruption.
|
24 |
+
|
25 |
| Model | Parameters |
|
26 |
| :-: | :-: |
|
27 |
| [thacio/ult5-pt-small](https://huggingface.co/thacio/ult5-pt-small) | 82.4M |
|
|
|
54 |
Os autores do UL2 apontam uma possivel diferença no resultado do fine-tune dependendo do modo ativado.
|
55 |
Porém, para o ult5-pt, não se notou diferença nos testes de benchmark.
|
56 |
|
57 |
+
*Fine-tunning is the recommended use for the model.*
|
58 |
|
59 |
+
*A tutorial (in portuguse) in notebook format for decoder and encoder-decoder (T5) model fine-tuning was provided: [Fine-tune Large Language Models](link here).*
|
60 |
|
61 |
+
*Span corruption modes can be activated by adding the prefixes '<|NLU|>' and '<|NLG|>' to the beginning of the text. The UL2 authors point out a possible difference in the fine-tuning result depending on the activated mode. However, for ult5-pt, no difference was noticed in benchmark tests.*
|
62 |
|
63 |
### Direct Use
|
64 |
|