README.md

---
base_model:
- GoToCompany/llama3-8b-cpt-sahabatai-v1-instruct
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
license: apache-2.0
language:
- id
datasets:
- psetialana/multi_session_chat-informal_indonesian-transformed
---

# Personalized Sahabat AI Llama 3.1 8 B

- **Developed by:** [Pradana Setialana](https://www.linkedin.com/in/psetialana/)

This model is a fine-tuned version of [GoToCompany/llama3-8b-cpt-sahabatai-v1-instruct](https://huggingface.co/GoToCompany/llama3-8b-cpt-sahabatai-v1-instruct) on [psetialana/multi_session_chat-informal_indonesian-transformed](https://huggingface.co/datasets/psetialana/multi_session_chat-informal_indonesian-transformed) dataset.

## Model description

This model can be used to personalize conversations and role-play based on the persona given with the prompt
```
Kamu adalah sahabat user. Kamu memiliki karakter PERSONA_ASSISTANT. User memiliki karakter PERSONA_USER. Kamu berperilaku sesuai PERSONA_ASSISTANT dan menyesuaikan responmu sesuai PERSONA_USER.

PERSONA_ASSISTANT:
{assistant_persona}

PERSONA_USER:
{user_persona}
```

## Training procedure

### LoRA config

The following lora config were used during training:
- alpha: 16
- r: 16
- droput: 0
- modules: "q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-4
- optimizer: adamw_8bit

### Training results

[TensorBoard](../../tensorboard)