license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
language:
- en
- he
library_name: transformers
Hebrew-Gemma-11B-Instruct
- Base Model: Hebrew-Gemma-11B
- Instruct Model: Hebrew-Gemma-11B-Instruct
The Hebrew-Gemma-11B-Instruct Large Language Model (LLM) is a instruct fine-tuned version of the Hebrew-Gemma-11B generative text model using a variety of conversation datasets.
It is continued pretrain of gemma-7b, extended to a larger scale and trained on 3B additional tokens of both English and Hebrew text data.
Instruction format
This format must be strictly respected, otherwise the model will generate sub-optimal outputs.
<bos><start_of_turn>user
Write a hello world program<end_of_turn>
<start_of_turn>model
Here is a simple hellow world program<end_of_turn>
<eos>
Each turn is preceded by a delimiter and then the role of the entity (either user, for content supplied by the user, or model for LLM responses). Turns finish with the token. You can follow this format to build the prompt manually, if you need to do it without the tokenizer's chat template.
A simple example:
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "Hebrew-Gemma-11B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="cuda")
chat = [
{ "role": "user", "content": "讻转讜讘 拽讜讚 驻砖讜讟 讘驻讬讬转讜谉 砖诪讚驻讬住 诇诪住讱 讗转 讛转讗专讬讱 砖诇 讛讬讜诐" },
]
prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
Terms of Use
As an extention of Gemma-7B, this model is subject to the original license and terms of use by Google.
Benchmark Results
- Coming Soon!
Notice
Hebrew-Gemma-11B is a pretrained base model and therefore does not have any moderation mechanisms.
Author
Trained by Yam Peleg.