Commencis-LLM

Commencis LLM is a generative model based on the Mistral 7B model. The base model adapts Mistral 7B to Turkish Banking specifically by training on a diverse dataset obtained through various methods, encompassing general Turkish and banking data.

Model Description

Training Details

Alignment phase consists of two stages: supervised fine-tuning (SFT) and Reward Modeling with Reinforcement learning from human feedback (RLHF).

The SFT phase was done on the a mixture of synthetic datasets generated from comprehensive banking dictionary data, synthetic datasets generated from banking-based domain and sub-domain headings, and derived from the CulturaX Turkish dataset by filtering. It was trained with three epochs. We used a learning rate 2e-5, lora rank 64 and maximum sequence length 1024 tokens.

Usage

Suggested Inference Parameters

  • Temperature: 0.5
  • Repetition penalty: 1.0
  • Top-p: 0.9
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

class TextGenerationAssistant:
    def __init__(self, model_id:str):
        self.tokenizer = AutoTokenizer.from_pretrained(model_id)
        self.model = AutoModelForCausalLM.from_pretrained(model_id, device_map='auto',load_in_8bit=True,load_in_4bit=False)
        self.pipe = pipeline("text-generation", 
                             model=self.model, 
                             tokenizer=self.tokenizer,
                             device_map="auto",
                             max_new_tokens=1024, 
                             return_full_text=True,
                             repetition_penalty=1.0
                            )

        self.sampling_params = dict(do_sample=True, temperature=0.5, top_k=50, top_p=0.9)
        self.system_prompt = "Sen yardımcı bir asistansın. Sana verilen talimat ve girdilere en uygun cevapları üreteceksin. \n\n\n"

    def format_prompt(self, user_input):
        return "[INST] " + self.system_prompt + user_input + " [/INST]"

    def generate_response(self, user_query):
        prompt = self.format_prompt(user_query)
        outputs = self.pipe(prompt, **self.sampling_params)
        return outputs[0]["generated_text"].split("[/INST]")[1].strip()


assistant = TextGenerationAssistant(model_id="Commencis/Commencis-LLM")

# Enter your query here.
user_query = "Faiz oranı yükseldiğinde kredi maliyetim nasıl etkilenir?"
response = assistant.generate_response(user_query)
print(response)

Chat Template

from transformers import AutoTokenizer
import transformers
import torch

model = "Commencis/Commencis-LLM"
messages = [{"role": "user", "content": "Faiz oranı yükseldiğinde kredi maliyetim nasıl etkilenir?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=1024, do_sample=True, temperature=0.5, top_k=50, top_p=0.9)
print (outputs[0]["generated_text"].split("[/INST]")[1].strip())

Quantized Models:

GGUF: https://huggingface.co/Commencis/Commencis-LLM-GGUF

Bias, Risks, and Limitations

Like all LLMs, Commencis-LLM has certain limitations:

  • Hallucination: Model may sometimes generate responses that contain plausible-sounding but factually incorrect or irrelevant information.
  • Code Switching: The model might unintentionally switch between languages or dialects within a single response, affecting the coherence and understandability of the output.
  • Repetition: The Model may produce repetitive phrases or sentences, leading to less engaging and informative responses.
  • Coding and Math: The model's performance in generating accurate code or solving complex mathematical problems may be limited.
  • Toxicity: The model could inadvertently generate responses containing inappropriate or harmful content.
Downloads last month
2,647
Safetensors
Model size
7.24B params
Tensor type
BF16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for Commencis/Commencis-LLM

Finetuned
(140)
this model
Quantizations
3 models

Dataset used to train Commencis/Commencis-LLM

Spaces using Commencis/Commencis-LLM 6