Update README.md

README.md

@@ -1,3 +1,83 @@
 ---
 license: mit
+datasets:
+- squad_v2
+- squad
+language:
+- en
+library_name: transformers
+pipeline_tag: question-answering
+tags:
+- question-answering
+- squad
+- squad_v2
+- t5
 ---
+
+# flan-t5-base for Extractive QA
+
+This is the [flan-t5-base](https://huggingface.co/google/flan-t5-base) model, fine-tuned using the [SQuAD2.0](https://huggingface.co/datasets/squad_v2) dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering.
+
+**NOTE:** The `<cls>` token must be manually added to the beginning of the question for this model to work properly.
+It uses the `<cls>` token to be able to make "no answer" predictions.
+The t5 tokenizer does not automatically add this special token which is why it is added manually.
+
+## Overview
+**Language model:** flan-t5-base  
+**Language:** English  
+**Downstream-task:** Extractive QA  
+**Training data:** SQuAD 2.0  
+**Eval data:** SQuAD 2.0  
+**Infrastructure**: 1x NVIDIA 3070  
+
+## Model Usage
+```python
+import torch
+from transformers import(
+  AutoModelForQuestionAnswering,
+  AutoTokenizer,
+  pipeline
+)
+model_name = "sjrhuschlee/flan-t5-base-squad2"
+
+# a) Using pipelines
+nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
+qa_input = {
+'question': f'{nlp.tokenizer.cls_token}Where do I live?',  # '<cls>Where do I live?'
+'context': 'My name is Sarah and I live in London'
+}
+res = nlp(qa_input)
+# {'score': 0.984, 'start': 30, 'end': 37, 'answer': ' London'}
+
+# b) Load model & tokenizer
+model = AutoModelForQuestionAnswering.from_pretrained(model_name)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+question = f'{tokenizer.cls_token}Where do I live?'  # '<cls>Where do I live?'
+context = 'My name is Sarah and I live in London'
+encoding = tokenizer(question, context, return_tensors="pt")
+start_scores, end_scores = model(
+  encoding["input_ids"],
+  attention_mask=encoding["attention_mask"],
+  return_dict=False
+)
+
+all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0].tolist())
+answer_tokens = all_tokens[torch.argmax(start_scores):torch.argmax(end_scores) + 1]
+answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
+# 'London'
+```
+
+## Metrics
+
+```bash
+# Squad v2
+
+# Squad
+```
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training: