premai-io

/

prem1b-sql-fp16-gguf

like 0

Follow

Prem 24

How to Prompt this model from a Python library such as llama_cpp_python. Please provide example

Probably you have just to use following code

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="premai-io/prem1b-sql-fp16-gguf",
    filename="unsloth.F16.gguf",
)

llm.create_chat_completion(
    messages = "No input example has been defined for this model task."
)