library_name: transformers
tags:
- Text Generation
- gaudi
- alpaca
- llm
- optimum-habana
- gaudi2
- llama
- llama3
- TextGeneration
license: apache-2.0
datasets:
- tatsu-lab/alpaca
language:
- en
Model Card for Model ID
This model was fine-tuned from the VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
Model Details
Model Description
The gopalakrishnan-d/Llama3-8b-VAGO-Gaudi-Alpaca-v1 model is a fine-tuned variant of the Llama3 architecture with 8 billion parameters. This version has been specifically enhanced for better performance on diverse language tasks, utilizing the Gaudi 2 Accelerator to optimize the training process.
- Hardware Type: Intel Gaudi2 Accelerator
- Cloud Provider: Intel® Tiber™ Developer Cloud
- Developed by: gopalakrishnan-d
- Model type: Fine-Tuned LLM
- Language(s) (NLP): English
- License: Apache 2.0 License
- Finetuned from model [optional]: VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
Uses
Conversational Agents
Description: Develop advanced chatbots or virtual assistants for customer service, support, and interactive help desks. Content Creation
Description: Assist in generating high-quality written content, including articles, marketing copy, and creative writing. Educational Tools
Description: Create interactive educational tools for tutoring, providing explanations, and generating study materials. Instruction-Based Applications
Description: Implement systems that follow and execute detailed user commands, such as workflow automation tools and interactive guides. Personalized Recommendations
Description: Provide tailored suggestions for products, services, or content based on user preferences and interactions.
Direct Use
Customer Service Chatbots Content Generation Tools Educational Tutoring Systems Workflow Automation Systems Personalized Recommendation Engines
Training Data
tatsu-lab/alpaca
Training Hyperparameters
- learning_rate: 5e-06 (Low Rate)
- train_batch_size: 8
- eval_batch_size: 8
- seed: 100
- gradient_accumulation_steps: 1
- optimizer: Adam
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.05
- lora_rank=16
- lora_alpha=32
Evaluation
epoch = 3.0 eval_accuracy = 0.6859 eval_loss = 1.3052