|
--- |
|
license: mit |
|
datasets: |
|
- ELiRF/dacsa |
|
- projecte-aina/CATalog |
|
language: |
|
- ca |
|
- en |
|
base_model: |
|
- openai-community/gpt2 |
|
- openai-community/gpt2-medium |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
# GPT-2 Medium Catalan-English Model |
|
|
|
The model is still being trained, and I will be making updates. Please do not expect great results just yet. 😀 |
|
|
|
## Model Overview |
|
This model is a GPT-2 Medium architecture trained **from scratch**, meaning it does not inherit any weights from existing models. It has been trained using **Catalan** dataset, specifically **ELiRF/dacsa** and **projecte-aina/CATalog**. |
|
|
|
## License and Usage |
|
This model is **free to use** under the MIT license. However, proper credit must be given when using it in research, applications, or any derived work. |
|
|
|
## Tokenizer |
|
The model utilizes a **52,000-token vocabulary**, using gpt2 config, specifically trained to handle Catalan, the tokenizer is also available in "Marxx01/gpt2-catalan-tokenizer". |
|
|
|
## How to Use |
|
To use this model for text generation, you can load it with the `transformers` library as follows: |
|
|
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
model_name = "Marxx01/gpt2_catalan" |
|
model = AutoModelForCausalLM.from_pretrained(model_name) |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
|
text = "El president de la generalitat va dir " |
|
inputs = tokenizer(text, return_tensors="pt") |
|
|
|
outputs = model.generate( |
|
**inputs, |
|
do_sample = True, |
|
max_length=150, |
|
temperature=0.7, |
|
top_p=0.8, |
|
top_k=1000, |
|
no_repeat_ngram_size=2, |
|
num_return_sequences=1 |
|
) |
|
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
