license: apache-2.0 | |
tags: | |
- Gemma Sprint | |
- GDE | |
- Google Cloud Champions Innovator | |
- Google Cloud Research Innovator | |
creator: Rabimba | |
# Gemma-rk-master1 | |
The strength of this model is | |
- Coding | |
- Poetic | |
- DPO Trained | |
## 💻 Usage | |
```python | |
!pip install -qU transformers bitsandbytes accelerate | |
from transformers import AutoTokenizer | |
import transformers | |
import torch | |
model = "rabimba/Gemma-rk-master1" | |
tokenizer = AutoTokenizer.from_pretrained(model) | |
pipeline = transformers.pipeline( | |
"text-generation", | |
model=model, | |
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True}, | |
) | |
messages = [{"role": "user", "content": "Explain what a LLM is in less than 100 words."}] | |
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) | |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) | |
print(outputs[0]["generated_text"]) | |
``` |