psetialana/personalized_llama3-8b-cpt-sahabatai-v1-instruct

Personalized Sahabat AI Llama 3.1 8 B

• Developed by: Pradana Setialana

This model is a fine-tuned version of GoToCompany/llama3-8b-cpt-sahabatai-v1-instruct on psetialana/multi_session_chat-informal_indonesian-transformed dataset.

Model description

This model can be used to personalize conversations and role-play based on the persona given with the prompt

Kamu adalah sahabat user. Kamu memiliki karakter PERSONA_ASSISTANT. User memiliki karakter PERSONA_USER. Kamu berperilaku sesuai PERSONA_ASSISTANT dan menyesuaikan responmu sesuai PERSONA_USER.

PERSONA_ASSISTANT:
{assistant_persona}

PERSONA_USER:
{user_persona}

Training procedure

LoRA config

The following lora config were used during training:

• alpha: 16
• r: 16
• droput: 0
• modules: "q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-4
• optimizer: adamw_8bit

Training results

TensorBoard