README.md

---
license: gemma
---
# BgGPT-Gemma-2-9B-IT-v1.0-GGUF

BgGPT is distributed under [Gemma Terms of Use](https://ai.google.dev/gemma/terms).

This repo contains the GGUF format model files for [INSAIT-Institute/BgGPT-Gemma-2-9B-IT-v1.0](https://huggingface.co/INSAIT-Institute/BgGPT-Gemma-2-9B-IT-v1.0).

## Quick Start using Python

Install the required package:
```bash
pip install llama-cpp-python
```

Example chat completion:

```python
from llama_cpp import Llama

llm = Llama(
    model_path="path/to/your/model.gguf",
    n_ctx=8192,
    penalize_nl=False
)

messages = [{"role": "user", "content": "Кога е основан Софийският университет?"}]
response = llm.create_chat_completion(
    messages=messages,
    max_tokens=2048,        # Choose maximum generated tokens
    temperature=0.1,
    top_p=0.9,
    repeat_penalty=1.0,
    stop=["<eos>", "<end_of_turn>"]
)


```

Example normal completion:

```python
from llama_cpp import Llama

llm = Llama(
    model_path="path/to/your/model.gguf",
    n_ctx=8192,
    penalize_nl=False
)

prompt = "<start_of_turn>user\nКога е основан Софийският университет?<end_of_turn>\n<start_of_turn>model\n"
response = llm(
    prompt,
    max_tokens=2048,        # Choose maximum generated tokens
    temperature=0.1,
    top_p=0.9,
    repeat_penalty=1.0,
    stop=["<eos>","<end_of_turn>"]
)
```