Update README.md
Browse files
README.md
CHANGED
@@ -7,11 +7,11 @@ license: mit
|
|
7 |
pipeline_tag: text-generation
|
8 |
---
|
9 |
|
10 |
-
# DeBERTa
|
11 |
|
12 |
This is [**deberta-v2-xxlarge**](https://huggingface.co/microsoft/deberta-v2-xxlarge) updated to implement the `AutoModelForCausalLM` class, enabling it to generate text. This implementation is based on our paper "BERTs are Generative In-Context Learners".
|
13 |
|
14 |
-
This repository also fixes three bugs in the original HF implementation of DeBERTa:
|
15 |
1. We fixed the incorrect name of the output embedding weights in the checkpoint file;
|
16 |
2. We fixed the implementation of the enhanced mask decoder (EMD), based on [the original GitHub repository](https://github.com/microsoft/DeBERTa);
|
17 |
3. We clamp the positional embeddings so that they work with long sequence lengths.
|
|
|
7 |
pipeline_tag: text-generation
|
8 |
---
|
9 |
|
10 |
+
# DeBERTa (1.4B) fixed version
|
11 |
|
12 |
This is [**deberta-v2-xxlarge**](https://huggingface.co/microsoft/deberta-v2-xxlarge) updated to implement the `AutoModelForCausalLM` class, enabling it to generate text. This implementation is based on our paper "BERTs are Generative In-Context Learners".
|
13 |
|
14 |
+
This repository also fixes three bugs in [the original HF implementation of DeBERTa](https://huggingface.co/microsoft/deberta-v2-xxlarge):
|
15 |
1. We fixed the incorrect name of the output embedding weights in the checkpoint file;
|
16 |
2. We fixed the implementation of the enhanced mask decoder (EMD), based on [the original GitHub repository](https://github.com/microsoft/DeBERTa);
|
17 |
3. We clamp the positional embeddings so that they work with long sequence lengths.
|