|
--- |
|
library_name: transformers |
|
base_model: P0x0/Epos-8b |
|
tags: |
|
- llama-cpp |
|
- gguf-my-repo |
|
license: llama3.1 |
|
--- |
|
|
|
# Triangle104/Epos-8b-Q4_K_S-GGUF |
|
This model was converted to GGUF format from [`P0x0/Epos-8b`](https://huggingface.co/P0x0/Epos-8b) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space. |
|
Refer to the [original model card](https://huggingface.co/P0x0/Epos-8b) for more details on the model. |
|
|
|
--- |
|
Model details: |
|
- |
|
Epos-8B is a fine-tuned version of the base model Llama-3.1-8B |
|
from Meta, optimized for storytelling, dialogue generation, and |
|
creative writing. The model specializes in generating rich narratives, |
|
immersive prose, and dynamic character interactions, making it ideal for |
|
creative tasks. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Model Details |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Model Description |
|
|
|
|
|
|
|
|
|
Epos-8B is an 8 billion parameter language model fine-tuned for |
|
storytelling and narrative tasks. Inspired by the grandeur of epic |
|
tales, it is designed to produce high-quality, engaging content that |
|
evokes the depth and imagination of ancient myths and modern |
|
storytelling traditions. |
|
|
|
|
|
Developed by: P0x0 |
|
Funded by: P0x0 |
|
Shared by: P0x0 |
|
Model type: Transformer-based Language Model |
|
Language(s) (NLP): Primarily English |
|
License: Apache 2.0 |
|
Finetuned from model: meta-llama/Llama-3.1-8B |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Model Sources |
|
|
|
|
|
|
|
|
|
Repository: Epos-8B on Hugging Face |
|
GGUF Repository: Epos-8B-GGUF (TO BE ADDED) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Uses |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Direct Use |
|
|
|
|
|
|
|
|
|
Epos-8B is ideal for: |
|
|
|
|
|
Storytelling: Generate detailed, immersive, and engaging narratives. |
|
Dialogue Creation: Create realistic and dynamic character interactions for stories or games. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
How to Get Started with the Model |
|
|
|
|
|
|
|
|
|
To run the quantized version of the model, you can use KoboldCPP, which allows you to run quantized GGUF models locally. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Steps: |
|
|
|
|
|
|
|
|
|
Download KoboldCPP. |
|
Follow the setup instructions provided in the repository. |
|
Download the GGUF variant of Epos-8B from Epos-8B-GGUF. |
|
Load the model in KoboldCPP and start generating! |
|
|
|
|
|
Alternatively, integrate the model directly into your code with the following snippet: |
|
|
|
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("P0x0/Epos-8B") |
|
model = AutoModelForCausalLM.from_pretrained("P0x0/Epos-8B") |
|
|
|
input_text = "Once upon a time in a distant land..." |
|
inputs = tokenizer(input_text, return_tensors="pt") |
|
outputs = model.generate(**inputs) |
|
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
|
--- |
|
## Use with llama.cpp |
|
Install llama.cpp through brew (works on Mac and Linux) |
|
|
|
```bash |
|
brew install llama.cpp |
|
|
|
``` |
|
Invoke the llama.cpp server or the CLI. |
|
|
|
### CLI: |
|
```bash |
|
llama-cli --hf-repo Triangle104/Epos-8b-Q4_K_S-GGUF --hf-file epos-8b-q4_k_s.gguf -p "The meaning to life and the universe is" |
|
``` |
|
|
|
### Server: |
|
```bash |
|
llama-server --hf-repo Triangle104/Epos-8b-Q4_K_S-GGUF --hf-file epos-8b-q4_k_s.gguf -c 2048 |
|
``` |
|
|
|
Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well. |
|
|
|
Step 1: Clone llama.cpp from GitHub. |
|
``` |
|
git clone https://github.com/ggerganov/llama.cpp |
|
``` |
|
|
|
Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux). |
|
``` |
|
cd llama.cpp && LLAMA_CURL=1 make |
|
``` |
|
|
|
Step 3: Run inference through the main binary. |
|
``` |
|
./llama-cli --hf-repo Triangle104/Epos-8b-Q4_K_S-GGUF --hf-file epos-8b-q4_k_s.gguf -p "The meaning to life and the universe is" |
|
``` |
|
or |
|
``` |
|
./llama-server --hf-repo Triangle104/Epos-8b-Q4_K_S-GGUF --hf-file epos-8b-q4_k_s.gguf -c 2048 |
|
``` |
