library_name: transformers
tags: []
Model Description
Llama-3.2-1B-finetuned-generalQA-peft-4bit is a fine-tuned version of the Llama-3.2-1B model, specialized for general question-answering tasks. The model has been fine-tuned using Low-Rank Adaptation (LoRA) with 4-bit quantization, making it efficient for deployment on resource-constrained hardware. Model Architecture
Base Model: Llama-3.2-1B
Parameters: Approximately 1 Billion
Quantization: 4-bit using the bitsandbytes library
Fine-tuning Method: PEFT with LoRA
Training Data
The model was fine-tuned on the Databricks Dolly 15k Subset for General QA dataset. This dataset is a subset focusing on general question-answering tasks, derived from the larger Databricks Dolly 15k dataset.
Training Procedure
Fine-tuning Configuration:
LoRA Rank (r): 8
LoRA Alpha: 16
LoRA Dropout: 0.5
Number of Epochs: 30
Batch Size: 2 (per device)
Learning Rate: 2e-5
Evaluation Strategy: Evaluated at each epoch
Optimizer: AdamW
Mixed Precision: FP16
Hardware Used: [Specify hardware if known, e.g., "Single NVIDIA A100 GPU"]
Libraries:
transformers
datasets
peft
bitsandbytes
trl
evaluate
Intended Use
The model is intended for generating informative answers to general questions. It can be integrated into applications such as chatbots, virtual assistants, educational tools, and information retrieval systems.
Limitations and Biases
Knowledge Cutoff: The model's knowledge is limited to the data it was trained on. It may not have information on events or developments that occurred after the dataset was created. Accuracy: While the model strives to provide accurate answers, it may occasionally produce incorrect or nonsensical responses. Always verify critical information from reliable sources. Biases: The model may inherit biases present in the training data. Users should be cautious and critically evaluate the model's outputs, especially in sensitive contexts.
Acknowledgements
Base Model: Meta AI's Llama-3.2-1B Dataset: Databricks Dolly 15k Subset for General QA Libraries Used: Transformers PEFT TRL BitsAndBytes