Create README.md

README.md

@@ -0,0 +1,53 @@
+---
+license: llama2
+datasets:
+- indiejoseph/ted-transcriptions-cantonese
+- indiejoseph/wikipedia-zh-yue-qa
+- indiejoseph/wikipedia-zh-yue-summaries
+- indiejoseph/ted-translation-zhhk-zhcn
+- OpenAssistant/oasst1
+language:
+- yue
+---
+
+# Cantonese Llama 2 7b v1
+
+## Model Introduction
+This model has been fine-tuned on [cantonese-llama-2-7b](https://huggingface.co/indiejoseph/cantonese-llama-2-7b), which is a second pretrained model based on Meta's llama2
+The fine-tuning process utilized a dataset consisting of [OpenAssistant/oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1)(with all Simplified Chinese removed),[indiejoseph/ted-transcriptions-cantonese](https://huggingface.co/datasets/indiejoseph/ted-transcriptions-cantonese), [indiejoseph/wikipedia-zh-yue-qa](https://huggingface.co/datasets/indiejoseph/wikipedia-zh-yue-qa), [indiejoseph/wikipedia-zh-yue-summaries](https://huggingface.co/datasets/indiejoseph/wikipedia-zh-yue-summaries), [indiejoseph/ted-translation-zhhk-zhcn](https://huggingface.co/datasets/indiejoseph/ted-translation-zhhk-zhcn).
+
+## Usage
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("cantonese-llama-2-7b-oasst-v1/", device_map="auto")
+tokenizer = AutoTokenizer.from_pretrained("cantonese-llama-2-7b-oasst-v1/")
+
+template = """A chat between a curious user and an artificial intelligence assistant.
+The assistant gives helpful, detailed, and polite answers to the user's questions.
+
+Human: {}
+
+Assistant: 
+"""
+
+tokenizer.pad_token = "[PAD]"
+tokenizer.padding_side = "left"
+
+def inference(input_texts):
+    inputs = tokenizer([template.format(text) for text in input_texts], return_tensors="pt", padding=True, truncation=True, max_length=512).to('cuda')
+
+    # Generate
+    generate_ids = model.generate(**inputs, max_new_tokens=512)
+    outputs = tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)
+    outputs = [out.split('Assistant:')[1].strip() for out in outputs]
+    
+    return outputs
+
+
+print(inference("香港現任特首係邊個？"))
+# Output: 香港現任特首係李家超。
+
+print(inference("2019年香港發生咗咩事？"))
+# Output: 2019年香港發生咗反修例運動。
+```