metadata
library_name: transformers
tags:
- text-generation-inference
datasets:
- cosmos_qa
Model Card for Model ID
Finetuned Phi2 model on the Cosmos QA dataset using PEFT and QLoRA for commonsense-based MCQ reading comprehension.
Model Details
Model Description
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- Developed by: Rajdeep Agrawal (https://huggingface.co/raj26000)
- Model type: Causal Language Model for Text Generation (https://huggingface.co/microsoft/phi-2)
- Finetuned from model [optional]: https://huggingface.co/microsoft/phi-2
Uses
Direct Use
The following code loads the finetuned adapter weights and merges them with the original frozen parameters to create the full model version.
lora_model = AutoPeftModelForCausalLM.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', device_map='auto', trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', trust_remote_code=True)
model = lora_model.merge_and_unload()
How to Get Started with the Model
Use the code below to get started with the model.
from datasets import load_dataset
from transformers import BitsAndBytesConfig, AutoTokenizer, AutoModelForCausalLM, TrainingArguments
from peft import AutoPeftModelForCausalLM
from tqdm import tqdm
class Inference:
def __init__(self):
self.data = load_dataset('cosmos_qa')
self.lora_model = AutoPeftModelForCausalLM.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', trust_remote_code=True)
self.tokenizer = AutoTokenizer.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', trust_remote_code=True)
self.model = self.lora_model.merge_and_unload()
self.test_data = self.data['test'].map(self.instruction_prompt_test)
def instruction_prompt_test(self, example):
instruction = f"""
Given a context and a question asked based on it, select the best answer from among the four provided choices deonted by 0. , 1. , 2. , 3.
Note that this would require reading between the lines over people's everyday narratives. The question could be based on the likely causes or effects of events and may require reasoning beyond the exact text spans in the context.\n
CONTEXT: {example['context']}\n
QUESTION: {example['question']}\n
CHOICES:
0. {example['answer0']}\n
1. {example['answer1']}\n
2. {example['answer2']}\n
3. {example['answer3']}\n
Answer:
"""
return {'text': instruction}
def instruct_predict(self, prompt):
tokenized_text = self.tokenizer(prompt, return_tensors='pt')
output = self.model(input_ids=tokenized_text['input_ids'].cuda(), attention_mask=tokenized_text['attention_mask'].cuda())
logits = output.logits[0][-1]
choices = [' 0', ' 1', ' 2', ' 3']
ans_logits = sorted([(logits[self.tokenizer(ans)['input_ids'][0]], ans.strip()) for ans in choices], key=lambda x: x[0], reverse=True)
pred_choice = ans_logits[0][1]
return pred_choice
def predict_sample(self, context, question, answer0, answer1, answer2, answer3):
example = {'context': context, 'question': question, 'answer0': answer0, 'answer1': answer1, 'answer2': answer2, 'answer3': answer3}
instruction = self.instruction_prompt_test(example)['text']
pred_choice = self.instruct_predict(instruction)
return pred_choice
infer = Inference()
infer.predict_sample(context='I posted a moment ago regarding a girl i 'd asked out at work and got some good advice . Basically , I asked this girl out at work and she said she would like to do something , but work made it difficult , although she did reinterate that she wanted to do something . This was a couple of weeks back now .',
question='Why do I keep saying that she reiterated that she wanted to do something ?',
answer0='Because I wanted to show off that she wanted me .',
answer1='I wanted to let people know she loved me .',
answer2='To let them know that she actually showed interest in me .',
answer3='To make sure people knew that she was mine .'
)
Training Details
Training Data
https://huggingface.co/datasets/cosmos_qa
Training Procedure
The model was trained using the HuggingFace Supervised Finetuning Trainer (SFT) with LoRA and 4-bit quantization offered by BitsAndBytes. An instructional prompt was created for each datapoint in the dataset and the model was trained over 2 epochs.
Preprocessing [optional]
The following instruction prompt was used to format the dataset before training:
def instruction_prompt(example):
instruction = f"""
Given a context and a question asked based on it, select the best answer from among the four provided choices deonted by 0. , 1. , 2. , 3.
Note that this would require reading between the lines over people's everyday narratives. The question could be based on the likely causes or effects of events and may require reasoning beyond the exact text spans in the context.\n
CONTEXT: {example['context']}\n
QUESTION: {example['question']}\n
CHOICES:
0. {example['answer0']}\n
1. {example['answer1']}\n
2. {example['answer2']}\n
3. {example['answer3']}\n
Answer: {example['label']}
"""
return {'text': instruction}
Training Hyperparameters
- Training regime:
- LoRA hyperparameters: r=16, lora_alpha=32, lora_dropout=0.05, target_modules=all-linear, bias=none, task_type=CAUSAL_LM
- Quantization: 4-bit Normal Float (nf4) quantization, double quantization, compute_dtype=torch.float16
- Training Args: learning_rate=3e-4, weight_decay=0.01, num_epochs=2, fp16=True, optim=adamw_torch_fused, gradient_accumulation_steps=2, gradient_checkpointing=True, neftune_noise_alpha=5
Speeds, Sizes, Times [optional]
Evaluation
Testing Data, Factors & Metrics
Testing Data
https://huggingface.co/datasets/cosmos_qa/viewer/default/test
Metrics
Accuracy metric on the Cosmos QA test dataset.
Results
85.85 %, ranked 18/88 on the Allen AI CosmosQA leaderboard.
Model Examination [optional]
Environmental Impact
Technical Specifications [optional]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
One 16GB P-100 GPU in a Kaggle environment.
Model Card Authors [optional]
Rajdeep Agrawal (https://huggingface.co/raj26000)