|
--- |
|
license: gpl-3.0 |
|
language: |
|
- gmh |
|
- de |
|
widget: |
|
- text: >- |
|
Ich Ott von Zintzndorff vergich mit dem offenn prief vnd tun chunt alln den |
|
leutn, di in sehnt oder hornt lesn, daz ich mit wolbedachtm mut vnd mit |
|
guetem rat vnd willn czu der czeit, do ich ez wol getun mochtt, den erbern |
|
herrn vnd fursten apt Englschalchn cze Seydensteten vnd sein gnants Gotshavs |
|
daselbs gancz vnd gar ledig sage vnd lazze aller der ansproch, die ich ... |
|
han auf seiner guter ains, des Schoephls lehn auf dem Graentleinsperg gnant |
|
in Groestner pharr gelegn, also, daz ich vnd alle mein erbn furbaz dhain |
|
ansprach dar vmb habn welln noch schulln in dhainn wegn, weder wenig noch |
|
vil. Vnd dar vmb czu eine steten vrchund gib ich dem vorgnantn Apt |
|
Englchalchn vnd seim wirdign Gotshaws cze Seydenstet den prief, versigelt |
|
mit meim des egnantn Ottn von Czintzndorff, vnd mit hern Dytrichs des |
|
Schenchn von Dobra anhangunden Insigeln, der das durch meinn willn cze |
|
gezeug der obgeschribn sach an den prief hat gehang. Das ist geschehn vnd |
|
der prief ist gebn nach Christs gepurd vber Drewtzehn hundert Jar, dar nach |
|
im Sibn vnd fumftzgisten Jar, am Eritag in den Phingstveyrtagn. |
|
--- |
|
|
|
# XLM-RoBERTa (base) Middle High German Charter Masked Language Model |
|
This model is a fine-tuned version of xlm-roberta-base on Middle High German (gmh; ISO 639-2; c. 1050–1500) charters of the [monasterium.net](https://www.icar-us.eu/en/cooperation/online-portals/monasterium-net/) data set. |
|
|
|
## Model description |
|
Please refer this model together with to the [XLM-RoBERTa (base-sized model)](https://huggingface.co/xlm-roberta-base) card or the paper [Unsupervised Cross-lingual Representation Learning at Scale by Conneau et al.](https://arxiv.org/abs/1911.02116) for additional information. |
|
|
|
## Intended uses & limitations |
|
This model can be used for sequence prediction tasks, i.e., fill-masks. |
|
|
|
## Training and evaluation data |
|
The model was fine-tuned using the Middle High German Monasterium charters. |
|
It was trained on a Tesla V100-SXM2-16GB GPU. |
|
|
|
## Training hyperparameters |
|
The following hyperparameters were used during training: |
|
- num_train_epochs: 15 |
|
- learning_rate: 2e-5 |
|
- weight-decay: 0,01 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- num_proc: 4 |
|
- block_size: 256 |
|
|
|
|
|
## Training results |
|
|
|
| Epoch | Training Loss | Validation Loss | |
|
|-------|---------------|------------------| |
|
| 1 | 2.423800 | 2.025645 | |
|
| 2 | 1.876500 | 1.700380 | |
|
| 3 | 1.702100 | 1.565900 | |
|
| 4 | 1.582400 | 1.461868 | |
|
| 5 | 1.506000 | 1.393849 | |
|
| 6 | 1.407300 | 1.359359 | |
|
| 7 | 1.385400 | 1.317869 | |
|
| 8 | 1.336700 | 1.285630 | |
|
| 9 | 1.301300 | 1.246812 | |
|
| 10 | 1.273500 | 1.219290 | |
|
| 11 | 1.245600 | 1.198312 | |
|
| 12 | 1.225800 | 1.198695 | |
|
| 13 | 1.214100 | 1.194895 | |
|
| 14 | 1.209500 | 1.177452 | |
|
| 15 | 1.200300 | 1.177396 | |
|
|
|
Perplexity: 3.25 |
|
|
|
## Updates |
|
- 2023-03-30: Upload |
|
|
|
|
|
## Citation |
|
Please cite the following papers when using this model. |
|
|
|
``` |
|
@misc{xlm-roberta-base-mhg-charter-mlm, |
|
title={xlm-roberta-base-mhg-charter-mlm}, |
|
author={Atzenhofer-Baumgartner, Florian}, |
|
year = { 2023 }, |
|
url = { https://huggingface.co/atzenhofer/xlm-roberta-base-mhg-charter-mlm }, |
|
publisher = { Hugging Face } |
|
} |
|
``` |
