README.md

metadata

license: apache-2.0
language:
  - vi
  - zh
metrics:
  - bleu
library_name: transformers
pipeline_tag: translation

viT5 for Sino-Vietnamese transliteration

Finetuned model from viT5 for Chinese MMORPG translation.

Model Description

Enhanced version from version 1.0 with larger dataset.

Uses

Default

Step 1: Map all Chinese word from original text to Sino-Vietnamese with map.json file

with open('/kaggle/input/chingchongdingdong/map.json', encoding = 'utf-8') as f:
    map = json.load(f)
global map
def mapping(text):
    for i in text:
        try:
            x = ' ' + map[i] + ' '
            text = text.replace(i, x)
        except:
            continue
    return text.strip()

Step 2: Load model and generate

Training Data

450k rows of system notifications, names and conversations translated from Chinese MMORPG games.