File size: 2,045 Bytes
863cf5c dcbbdbe 863cf5c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
---
language:
- en
datasets:
- c4
- squad
tags:
- text2text-generation
widget:
- text: "question: What is the atomic number for oxygen? context: Oxygen is a chemical element with symbol O and atomic number 8."
- text: "question: What is the chemical symbol of Oxygen? context: Oxygen is a chemical element with symbol O and atomic number 8."
license: apache-2.0
---
T5-small for QA
---
[Google's T5-small](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) pre-trained on the [C4](https://huggingface.co/datasets/c4) dataset, fine-tuned for Question-Answering on [SQuAD v2](https://huggingface.co/datasets/squad_v2) with the following hyperparameters:
```
optimizer=adamw_hf
learning_rate=3e-5
adam_beta1=0.9
adam_beta2=0.999
adam_epsilon=1e-08
num_train_epochs=2
per_device_train_batch_size=12
```
Usage
---
The input [context and question] has to be prepared in a specific way as follows:
```python
from transformers import pipeline
def prep_input(_context, _question):
return " ".join(["question:", _question.strip(), "context:", _context.strip()])
t5qa = pipeline("text2text-generation", "fgaim/t5-small-squad-v2")
context = """
Oxygen is a chemical element with symbol O and atomic number 8. It is a member of the chalcogen group on the periodic table and is a highly reactive nonmetal and oxidizing agent that readily forms compounds (notably oxides) with most elements. By mass, oxygen is the third-most abundant element in the universe, after hydrogen and helium. At standard temperature and pressure, two atoms of the element bind to form dioxygen, a colorless and odorless diatomic gas with the formula O.
"""
t5qa(prep_input(context, "How many atoms combine to form dioxygen?"))
# [{'generated_text': 'two'}]
t5qa(prep_input(context, "What element makes up almost half of the earth's crust by mass?"))
# [{'generated_text': 'oxygen'}]
t5qa(prep_input(context, "What are the most abundent elements of the universe by mass?"))
# [{'generated_text': 'hydrogen and helium'}]
```
|