--- library_name: transformers tags: - text-generation-inference datasets: - cosmos_qa --- # Model Card for Model ID Finetuned Phi2 model on the Cosmos QA dataset using PEFT and QLoRA for commonsense-based MCQ reading comprehension. ## Model Details ### Model Description This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. - **Developed by:** Rajdeep Agrawal (https://huggingface.co/raj26000) - **Model type:** Causal Language Model for Text Generation (https://huggingface.co/microsoft/phi-2) - **Finetuned from model [optional]:** https://huggingface.co/microsoft/phi-2 ## Uses ### Direct Use The following code loads the finetuned adapter weights and merges them with the original frozen parameters to create the full model version. ``` lora_model = AutoPeftModelForCausalLM.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', device_map='auto', trust_remote_code=True) tokenizer = AutoTokenizer.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', trust_remote_code=True) model = lora_model.merge_and_unload() ``` ## How to Get Started with the Model Use the code below to get started with the model. ``` from datasets import load_dataset from transformers import BitsAndBytesConfig, AutoTokenizer, AutoModelForCausalLM, TrainingArguments from peft import AutoPeftModelForCausalLM from tqdm import tqdm class Inference: def __init__(self): self.data = load_dataset('cosmos_qa') self.lora_model = AutoPeftModelForCausalLM.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', trust_remote_code=True) self.tokenizer = AutoTokenizer.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', trust_remote_code=True) self.model = self.lora_model.merge_and_unload() self.test_data = self.data['test'].map(self.instruction_prompt_test) def instruction_prompt_test(self, example): instruction = f""" Given a context and a question asked based on it, select the best answer from among the four provided choices deonted by 0. , 1. , 2. , 3. Note that this would require reading between the lines over people's everyday narratives. The question could be based on the likely causes or effects of events and may require reasoning beyond the exact text spans in the context.\n CONTEXT: {example['context']}\n QUESTION: {example['question']}\n CHOICES: 0. {example['answer0']}\n 1. {example['answer1']}\n 2. {example['answer2']}\n 3. {example['answer3']}\n Answer: """ return {'text': instruction} def instruct_predict(self, prompt): tokenized_text = self.tokenizer(prompt, return_tensors='pt') output = self.model(input_ids=tokenized_text['input_ids'].cuda(), attention_mask=tokenized_text['attention_mask'].cuda()) logits = output.logits[0][-1] choices = [' 0', ' 1', ' 2', ' 3'] ans_logits = sorted([(logits[self.tokenizer(ans)['input_ids'][0]], ans.strip()) for ans in choices], key=lambda x: x[0], reverse=True) pred_choice = ans_logits[0][1] return pred_choice def predict_sample(self, context, question, answer0, answer1, answer2, answer3): example = {'context': context, 'question': question, 'answer0': answer0, 'answer1': answer1, 'answer2': answer2, 'answer3': answer3} instruction = self.instruction_prompt_test(example)['text'] pred_choice = self.instruct_predict(instruction) return pred_choice infer = Inference() infer.predict_sample(context='I posted a moment ago regarding a girl i 'd asked out at work and got some good advice . Basically , I asked this girl out at work and she said she would like to do something , but work made it difficult , although she did reinterate that she wanted to do something . This was a couple of weeks back now .', question='Why do I keep saying that she reiterated that she wanted to do something ?', answer0='Because I wanted to show off that she wanted me .', answer1='I wanted to let people know she loved me .', answer2='To let them know that she actually showed interest in me .', answer3='To make sure people knew that she was mine .' ) ``` ## Training Details ### Training Data https://huggingface.co/datasets/cosmos_qa ### Training Procedure The model was trained using the HuggingFace Supervised Finetuning Trainer (SFT) with LoRA and 4-bit quantization offered by BitsAndBytes. An instructional prompt was created for each datapoint in the dataset and the model was trained over 2 epochs. #### Preprocessing [optional] The following instruction prompt was used to format the dataset before training: ``` def instruction_prompt(example): instruction = f""" Given a context and a question asked based on it, select the best answer from among the four provided choices deonted by 0. , 1. , 2. , 3. Note that this would require reading between the lines over people's everyday narratives. The question could be based on the likely causes or effects of events and may require reasoning beyond the exact text spans in the context.\n CONTEXT: {example['context']}\n QUESTION: {example['question']}\n CHOICES: 0. {example['answer0']}\n 1. {example['answer1']}\n 2. {example['answer2']}\n 3. {example['answer3']}\n Answer: {example['label']} """ return {'text': instruction} ``` #### Training Hyperparameters - **Training regime:** - LoRA hyperparameters: r=16, lora_alpha=32, lora_dropout=0.05, target_modules=all-linear, bias=none, task_type=CAUSAL_LM - Quantization: 4-bit Normal Float (nf4) quantization, double quantization, compute_dtype=torch.float16 - Training Args: learning_rate=3e-4, weight_decay=0.01, num_epochs=2, fp16=True, optim=adamw_torch_fused, gradient_accumulation_steps=2, gradient_checkpointing=True, neftune_noise_alpha=5 #### Speeds, Sizes, Times [optional] ## Evaluation ### Testing Data, Factors & Metrics #### Testing Data https://huggingface.co/datasets/cosmos_qa/viewer/default/test #### Metrics Accuracy metric on the Cosmos QA test dataset. ### Results 85.85 %, ranked 18/88 on the Allen AI CosmosQA leaderboard. ## Model Examination [optional] ## Environmental Impact ## Technical Specifications [optional] ### Model Architecture and Objective [More Information Needed] ### Compute Infrastructure One 16GB P-100 GPU in a Kaggle environment. ## Model Card Authors [optional] Rajdeep Agrawal (https://huggingface.co/raj26000)