FINGU-AI/L3-78b-Large-v1

Overview

FINGU-AI/L3-78b-Large-v1 is a powerful causal language model designed for a variety of natural language processing (NLP) tasks, including machine translation, text generation, and chat-based applications. This model is particularly useful for translating between languages, as well as supporting other custom NLP tasks through flexible input.

Example Usage

Installation

Make sure to install the required packages:

pip install torch transformers

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Model and Tokenizer
model_id = 'FINGU-AI/L3-78b-Large-v1'
model = AutoModelForCausalLM.from_pretrained(model_id, attn_implementation="sdpa", torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_id)
model.to('cuda')

# Input Messages for Translation
messages = [
    {"role": "system", "content": "translate korean to Uzbek"},
    {"role": "user", "content": """새로운 은행 계좌를 개설하는 절차는 다음과 같습니다:

1. 계좌 개설 목적과 신분 확인을 위한 서류 제출
2. 서류 검토 과정을 거치는 것
3. 고객님의 신원 확인 절차를 진행하는 것
4. 모든 절차가 완료되면 계좌 개설이 가능합니다.

계좌 개설을 원하시는 경우, 신분증과 함께 방문해 주시면 됩니다.
    """},
]

# Tokenize and Generate Response
input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to('cuda')

outputs = model.generate(
    input_ids,
    max_new_tokens=500,
    do_sample=True,
)

# Decode and Print the Translation
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))