README.md

---
library_name: transformers
tags:
- text-generation-inference
datasets:
- cosmos_qa
---

# Model Card for Model ID
Finetuned Phi2 model on the Cosmos QA dataset using PEFT and QLoRA for commonsense-based MCQ reading comprehension.
<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** Rajdeep Agrawal (https://huggingface.co/raj26000)
- **Model type:** Causal Language Model for Text Generation (https://huggingface.co/microsoft/phi-2)
- **Finetuned from model [optional]:** https://huggingface.co/microsoft/phi-2



## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
The following code loads the finetuned adapter weights and merges them with the original frozen parameters to create the full model version.
```
lora_model = AutoPeftModelForCausalLM.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', device_map='auto', trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', trust_remote_code=True)
model = lora_model.merge_and_unload()
```

## How to Get Started with the Model

Use the code below to get started with the model.

```
from datasets import load_dataset
from transformers import BitsAndBytesConfig, AutoTokenizer, AutoModelForCausalLM, TrainingArguments
from peft import AutoPeftModelForCausalLM
from tqdm import tqdm

class Inference:
    def __init__(self):
        self.data = load_dataset('cosmos_qa')
        self.lora_model = AutoPeftModelForCausalLM.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', trust_remote_code=True)
        self.tokenizer = AutoTokenizer.from_pretrained('raj26000/phi2-instruct-cosmosqa-qlora', trust_remote_code=True)
        self.model = self.lora_model.merge_and_unload()
        self.test_data = self.data['test'].map(self.instruction_prompt_test)


    def instruction_prompt_test(self, example):
        instruction = f"""
        Given a context and a question asked based on it, select the best answer from among the four provided choices deonted by 0. , 1. , 2. , 3.
        Note that this would require reading between the lines over people's everyday narratives. The question could be based on the likely causes or effects of events and may require reasoning beyond the exact text spans in the context.\n
        CONTEXT: {example['context']}\n
        QUESTION: {example['question']}\n
        CHOICES:
        0. {example['answer0']}\n
        1. {example['answer1']}\n
        2. {example['answer2']}\n
        3. {example['answer3']}\n
        Answer:
        """
        return {'text': instruction}

    def instruct_predict(self, prompt):
        tokenized_text = self.tokenizer(prompt, return_tensors='pt')
        output = self.model(input_ids=tokenized_text['input_ids'].cuda(), attention_mask=tokenized_text['attention_mask'].cuda())
        logits = output.logits[0][-1]
        choices = [' 0', ' 1', ' 2', ' 3']
        ans_logits = sorted([(logits[self.tokenizer(ans)['input_ids'][0]], ans.strip()) for ans in choices], key=lambda x: x[0], reverse=True)
        pred_choice = ans_logits[0][1]
        return pred_choice

    def predict_sample(self, context, question, answer0, answer1, answer2, answer3):
        example = {'context': context, 'question': question, 'answer0': answer0, 'answer1': answer1, 'answer2': answer2, 'answer3': answer3}
        instruction = self.instruction_prompt_test(example)['text']
        pred_choice = self.instruct_predict(instruction)
        return pred_choice


infer = Inference()
infer.predict_sample(context='I posted a moment ago regarding a girl i 'd asked out at work and got some good advice . Basically , I asked this girl out at work and she said she would like to do something , but work made it difficult , although she did reinterate that she wanted to do something . This was a couple of weeks back now .',
                     question='Why do I keep saying that she reiterated that she wanted to do something ?',
                     answer0='Because I wanted to show off that she wanted me .',
                     answer1='I wanted to let people know she loved me .',
                     answer2='To let them know that she actually showed interest in me .',
                     answer3='To make sure people knew that she was mine .'
)
```

## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

https://huggingface.co/datasets/cosmos_qa

### Training Procedure 

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
The model was trained using the HuggingFace Supervised Finetuning Trainer (SFT) with LoRA and 4-bit quantization offered by BitsAndBytes. An instructional prompt was created for each datapoint in the dataset and the model was trained over 2 epochs.

#### Preprocessing [optional]

The following instruction prompt was used to format the dataset before training:
```
def instruction_prompt(example):
    instruction = f"""
    Given a context and a question asked based on it, select the best answer from among the four provided choices deonted by 0. , 1. , 2. , 3.
    Note that this would require reading between the lines over people's everyday narratives. The question could be based on the likely causes or effects of events and may require reasoning beyond the exact text spans in the context.\n
    CONTEXT: {example['context']}\n
    QUESTION: {example['question']}\n
    CHOICES:
    0. {example['answer0']}\n
    1. {example['answer1']}\n
    2. {example['answer2']}\n
    3. {example['answer3']}\n
    Answer: {example['label']}    
    """
    return {'text': instruction}
```


#### Training Hyperparameters

- **Training regime:**
  - LoRA hyperparameters: r=16, lora_alpha=32, lora_dropout=0.05, target_modules=all-linear, bias=none, task_type=CAUSAL_LM
  - Quantization: 4-bit Normal Float (nf4) quantization, double quantization, compute_dtype=torch.float16
  - Training Args: learning_rate=3e-4, weight_decay=0.01, num_epochs=2, fp16=True, optim=adamw_torch_fused, gradient_accumulation_steps=2, gradient_checkpointing=True, neftune_noise_alpha=5

#### Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->



## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

https://huggingface.co/datasets/cosmos_qa/viewer/default/test



#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

Accuracy metric on the Cosmos QA test dataset.

### Results

85.85 %, ranked 18/88 on the Allen AI CosmosQA leaderboard.


## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->



## Technical Specifications [optional]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

One 16GB P-100 GPU in a Kaggle environment.


## Model Card Authors [optional]

Rajdeep Agrawal (https://huggingface.co/raj26000)