RichardErkhov commited on
Commit
6d794ff
·
verified ·
1 Parent(s): 6d4ad2e

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +89 -0
README.md ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ Myanmarsar-GPT - bnb 8bits
11
+ - Model creator: https://huggingface.co/simbolo-ai/
12
+ - Original model: https://huggingface.co/simbolo-ai/Myanmarsar-GPT/
13
+
14
+
15
+
16
+
17
+ Original model description:
18
+ ---
19
+ license: mit
20
+ language:
21
+ - my
22
+ pipeline_tag: text-generation
23
+ metrics:
24
+ - code_eval
25
+ library_name: transformers
26
+ tags:
27
+ - burmese
28
+ - gpt2
29
+ - pre-trained
30
+ ---
31
+
32
+
33
+ The Simbolo's Myanmarsar-GPT (it is not a chatbot but a text generation model which can be used to develop chatbot) is pre-trained on a dataset of 20,000 Burmese data and pre-trained using the GPT-2 architecture of MGPT Model. Its purpose is to serve as a foundational pre-trained model for the Burmese language, facilitating fine-tuning for specific applications of different tasks such as creative writing, chatbot, machine translation etc.
34
+
35
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6598b82502c4796342239a35/rFId3-xyzWW-juDq_er9k.jpeg)
36
+
37
+
38
+
39
+
40
+ ### How to use
41
+
42
+ ```python
43
+ from transformers import AutoTokenizer, AutoModelForCausalLM
44
+
45
+ tokenizer = AutoTokenizer.from_pretrained("Simbolo-Servicio/Myanmarsar-GPT")
46
+ model = AutoModelForCausalLM.from_pretrained("Simbolo-Servicio/Myanmarsar-GPT")
47
+
48
+ input_text = "ပညာရေး"
49
+ input_ids = tokenizer.encode(input_text, return_tensors='pt')
50
+ output = model.generate(input_ids, max_length=50)
51
+ print(tokenizer.decode(output[0], skip_special_tokens=True))
52
+ ```
53
+ ### Data
54
+ We use 20,000 Burmese sentences and most are from our open-source [data](https://huggingface.co/datasets/Simbolo-Servicio/wiki-burmese-sentences) which contains 100,000 sentences sourced from Wikipedia.
55
+
56
+ ### Contributors
57
+ Main Contributor: [Sa Phyo Thu Htet](https://github.com/SaPhyoThuHtet)
58
+ Wikipedia Data Crawling: Kaung Kaung Ko Ko, Phuu Pwint Thinzar Kyaing
59
+ Releasing the Model: Eithandaraung, Ye Yint Htut, Thet Chit Su, Naing Phyo Aung, Nyan Linn Phyo Zaw, Lynn Thu Kha
60
+
61
+ ### Acknowledgment
62
+ We extend our gratitude to the creators of the [mGPT-XL](https://huggingface.co/ai-forever/mGPT) models for their invaluable contribution to this project.
63
+ We want to thank everyone who has worked on the related works, especially [Minsithu](https://huggingface.co/jojo-ai-mst/MyanmarGPTT) and
64
+ [Dr. Wai Yan Nyein Naing](https://huggingface.co/WYNN747/Burmese-GPT)who initiated the work of gpt-2 model.
65
+ And We would like to thank Simbolo:Servico which is a branch of Simbolo under the company of Intello Tech for providing financial support.
66
+
67
+ ### Limitations and Bias
68
+ We have yet to investigate the potential bias inherent in this model thoroughly. Regarding transparency, it's important to note that the model is primarily trained on data from the Unicode Burmese(Myanmar) language.
69
+
70
+ ### References
71
+ 1. Jiang, Shengyi & Huang, Xiuwen & Cai, Xiaonan & Lin, Nankai. (2021). Pre-trained Models and Evaluation Data for the Myanmar Language. 10.1007/978-3-030-92310-5_52.
72
+ 2. Lin, N., Fu, Y., Chen, C., Yang, Z., & Jiang, S. (2021). LaoPLM: Pre-trained Language Models for Lao. ArXiv. /abs/2110.05896
73
+ 3. MinSithu, MyanmarGPT, https://huggingface.co/jojo-ai-mst/MyanmarGPT, 1.1-SweptWood
74
+ 4. Wai Yan Nyein Naing, WYNN747/Burmese-GPT, https://huggingface.co/WYNN747/Burmese-GPT
75
+ 5. Sai Htaung Kham, saihtaungkham/BurmeseRoBERTaCLM
76
+ 6. Shliazhko, O., Fenogenova, A., Tikhonova, M., Mikhailov, V., Kozlova, A., & Shavrina, T. (2022). MGPT: Few-Shot Learners Go Multilingual. ArXiv. /abs/2204.07580
77
+
78
+ ### How to Cite this work:
79
+ ### Cite As:
80
+ ```bibtex
81
+ @misc{myanmarsar-gpt,
82
+ author = {{Sa Phyo Thu Htet}},
83
+ title = {Myanmarsar GPT},
84
+ url = {https://huggingface.co/Simbolo-Servicio/Myanmarsar-GPT},
85
+ urldate = {2024-1-09},
86
+ date = {2024-1-09}
87
+ }
88
+ ```
89
+