|
--- |
|
license: cc-by-nc-sa-4.0 |
|
language: |
|
- hu |
|
- en |
|
tags: |
|
- translation |
|
- opennmt |
|
--- |
|
|
|
inference: false |
|
--- |
|
|
|
### Introduction |
|
|
|
Hungarian - English translation model that was trained on the [Hunglish2](http://mokk.bme.hu/resources/hunglishcorpus/) dataset using OpenNMT. |
|
|
|
### Usage |
|
|
|
Install the necessary dependencies: |
|
|
|
```bash |
|
pip3 install ctranslate2 pyonmttok |
|
``` |
|
|
|
Simple tokenization & translation using Python: |
|
|
|
|
|
```python |
|
import ctranslate2 |
|
import pyonmttok |
|
from huggingface_hub import snapshot_download |
|
model_dir = snapshot_download(repo_id="SZTAKI-HLT/opennmt-hu-en", revision="main") |
|
|
|
tokenizer=pyonmttok.Tokenizer(mode="none", sp_model_path = model_dir + "/sp_m.model") |
|
tokenized=tokenizer.tokenize("Hello világ") |
|
|
|
translator = ctranslate2.Translator(model_dir) |
|
translated = translator.translate_batch([tokenized[0]]) |
|
print(tokenizer.detokenize(translated[0][0]['tokens'])) |
|
``` |
|
|
|
|
|
## Citation |
|
|
|
If you use our model, please cite the following paper: |
|
``` |
|
|
|
@inproceedings{nagy2022syntax, |
|
title={Syntax-based data augmentation for Hungarian-English machine translation}, |
|
author={Nagy, Attila and Nanys, Patrick and Konr{\'a}d, Bal{\'a}zs Frey and Bial, Bence and {\'A}cs, Judit}, |
|
booktitle = {XVIII. Magyar Számítógépes Nyelvészeti Konferencia (MSZNY 2022)}, |
|
year={2022}, |
|
publisher = {Szegedi Tudományegyetem, Informatikai Intézet}, |
|
} |
|
|
|
``` |
