Update README.md

README.md

@@ -6,10 +6,11 @@ widget:
   example_title: Spelling Correction
 ---
 
-# Model Card for Model ID
+# Model Card for T5-Shona-SC
 
 <!-- Provide a quick summary of what the model is/does. -->
-
+<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/flan2_architecture.jpg"
+alt="drawing" width="600"/>
 
 
 ## Model Details
@@ -18,15 +19,10 @@ widget:
 
 <!-- Provide a longer summary of what this model is. -->
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Developed by:** [Thabolezwe Mabandla](http://www.linkedin.com/in/thabolezwe-mabandla-81a62a22b)
+- **Model type:** Language Model
+- **Language(s) (NLP):** Shona
+- **Finetuned from model [optional]:** [FLAN-T5](https://huggingface.co/google/flan-t5-small)
 
 ### Model Sources [optional]
 
@@ -39,107 +35,126 @@ This is the model card of a 🤗 transformers model that has been pushed on the
 ## Uses
 
 <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+> Correction of spelling errors in shona sentences or phrases.
 
 ### Direct Use
 
 <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+> Spelling correction
 
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+# Bias, Risks, and Limitations
 
-[More Information Needed]
+The information below in this section are copied from the model's [official model card](https://arxiv.org/pdf/2210.11416.pdf):
 
-### Recommendations
+> Language models, including Flan-T5, can potentially be used for language generation in a harmful way, according to Rae et al. (2021). Flan-T5 should not be used directly in any application, without a prior assessment of safety and fairness concerns specific to the application.
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+## Ethical considerations and risks
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+> Flan-T5 is fine-tuned on a large corpus of text data that was not filtered for explicit content or assessed for existing biases. As a result the model itself is potentially vulnerable to generating equivalently inappropriate content or replicating inherent biases in the underlying data.
 
 ## How to Get Started with the Model
 
 Use the code below to get started with the model.
 
-[More Information Needed]
+### Running the model on a CPU
 
-## Training Details
+<details>
+<summary> Click to expand </summary>
 
-### Training Data
+```python
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+from transformers import T5Tokenizer, T5ForConditionalGeneration
 
-[More Information Needed]
+tokenizer = T5Tokenizer.from_pretrained("thaboe01/t5-spelling-corrector")
+model = T5ForConditionalGeneration.from_pretrained("thaboe01/t5-spelling-corrector")
 
-### Training Procedure
+input_text = "Please correct the following sentence: ndaids kurnda kumba kwaco"
+input_ids = tokenizer(input_text, return_tensors="pt").input_ids
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+outputs = model.generate(input_ids)
+print(tokenizer.decode(outputs[0]))
+```
 
-#### Preprocessing [optional]
+</details>
 
-[More Information Needed]
+### Running the model on a GPU
 
+<details>
+<summary> Click to expand </summary>
 
-#### Training Hyperparameters
+```python
+# pip install accelerate
+from transformers import T5Tokenizer, T5ForConditionalGeneration
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+tokenizer = T5Tokenizer.from_pretrained("thaboe01/t5-spelling-corrector")
+model = T5ForConditionalGeneration.from_pretrained("thaboe01/t5-spelling-corrector", device_map="auto")
 
-#### Speeds, Sizes, Times [optional]
+input_text = "Please correct the following sentence: ndaids kurnda kumba kwaco"
+input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+outputs = model.generate(input_ids)
+print(tokenizer.decode(outputs[0]))
+```
 
-[More Information Needed]
+</details>
 
-## Evaluation
+### Running the model on a GPU using different precisions
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+#### FP16
 
-### Testing Data, Factors & Metrics
+<details>
+<summary> Click to expand </summary>
 
-#### Testing Data
+```python
+# pip install accelerate
+import torch
+from transformers import T5Tokenizer, T5ForConditionalGeneration
 
-<!-- This should link to a Dataset Card if possible. -->
+tokenizer = T5Tokenizer.from_pretrained("thaboe01/t5-spelling-corrector")
+model = T5ForConditionalGeneration.from_pretrained("thaboe01/t5-spelling-corrector", device_map="auto", torch_dtype=torch.float16)
 
-[More Information Needed]
+input_text = "Please correct the following sentence: ndaids kurnda kumba kwaco"
+input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
 
-#### Factors
+outputs = model.generate(input_ids)
+print(tokenizer.decode(outputs[0]))
+```
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+</details>
 
-[More Information Needed]
+#### INT8
 
-#### Metrics
+<details>
+<summary> Click to expand </summary>
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+```python
+# pip install bitsandbytes accelerate
+from transformers import T5Tokenizer, T5ForConditionalGeneration
+
+tokenizer = T5Tokenizer.from_pretrained("thaboe01/t5-spelling-corrector")
+model = T5ForConditionalGeneration.from_pretrained("thaboe01/t5-spelling-corrector", device_map="auto", load_in_8bit=True)
 
-[More Information Needed]
+input_text = "Please correct the following sentence: ndaids kurnda kumba kwaco"
+input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
 
-### Results
+outputs = model.generate(input_ids)
+print(tokenizer.decode(outputs[0]))
+```
 
-[More Information Needed]
+</details>
 
-#### Summary
 
+## Evaluation
 
+<!-- This section describes the evaluation protocols and provides the results. -->
 
-## Model Examination [optional]
+### Testing Metrics
 
-<!-- Relevant interpretability work for the model goes here -->
+#### Metrics
 
-[More Information Needed]
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+<img src=""
+alt="metrics" width="600"/>
 
 ## Environmental Impact
 
@@ -147,56 +162,14 @@ Use the code below to get started with the model.
 
 Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
+- **Hardware Type:** [T4 GPU x 2]
+- **Hours used:** [8]
+- **Cloud Provider:** [Kaggle]
 
 ## Model Card Authors [optional]
 
-[More Information Needed]
+Thabolezwe Mabandla
 
 ## Model Card Contact
 
-[More Information Needed]
+[email protected]