README.md

---
library_name: transformers
tags:
- Text Generation
- gaudi
- alpaca
- llm
- optimum-habana
- gaudi2
- llama
- llama3
- TextGeneration
license: apache-2.0
datasets:
- tatsu-lab/alpaca
language:
- en
---

# Model Card for Model ID

This model was fine-tuned from the VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct 

## Model Details

### Model Description

The gopalakrishnan-d/Llama3-8b-VAGO-Gaudi-Alpaca-v1 model is a fine-tuned variant of the Llama3 architecture with 8 billion parameters.
This version has been specifically enhanced for better performance on diverse language tasks, utilizing the Gaudi 2 Accelerator to optimize the training process.

- **Hardware Type:** Intel Gaudi2 Accelerator
- **Cloud Provider:** Intel® Tiber™ Developer Cloud
- **Developed by:** gopalakrishnan-d
- **Model type:** Fine-Tuned LLM
- **Language(s) (NLP):** English
- **License:** **Apache 2.0 License**
- **Finetuned from model [optional]:** VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct 

## Uses

Conversational Agents

Description: Develop advanced chatbots or virtual assistants for customer service, support, and interactive help desks.
Content Creation

Description: Assist in generating high-quality written content, including articles, marketing copy, and creative writing.
Educational Tools

Description: Create interactive educational tools for tutoring, providing explanations, and generating study materials.
Instruction-Based Applications

Description: Implement systems that follow and execute detailed user commands, such as workflow automation tools and interactive guides.
Personalized Recommendations

Description: Provide tailored suggestions for products, services, or content based on user preferences and interactions.
### Direct Use

Customer Service Chatbots
Content Generation Tools
Educational Tutoring Systems
Workflow Automation Systems
Personalized Recommendation Engines
    
### Training Data
    tatsu-lab/alpaca

#### Training Hyperparameters

    - learning_rate: 5e-06 (Low Rate)
    - train_batch_size: 8
    - eval_batch_size: 8
    - seed: 100
    - gradient_accumulation_steps: 1
    - optimizer: Adam 
    - lr_scheduler_type: linear
    - lr_scheduler_warmup_ratio: 0.05
    - lora_rank=16 
    - lora_alpha=32

## Evaluation

  epoch                       =        3.0
  eval_accuracy               =     0.6859
  eval_loss                   =     1.3052

### Results