File size: 365 Bytes
6fc5663 a749474 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
---
license: apache-2.0
---
To use with [LocalAI](https://localai.io) this is an example model definition:
```
name: WizardLM2
backend: transformers
parameters:
model: fakezeta/Not-WizardLM-2-7B-ov-int8
context_size: 8192
threads: 6
f16: true
type: OVModelForCausalLM
prompt_cache_path: "cache"
prompt_cache_all: true
template:
use_tokenizer_template: true
``` |