File size: 333 Bytes
72268ee |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
Codellama-2-7b-Miniguanaco-Mistral-AR-Q5_K_M.gguf$:
loader: llama.cpp
cpu: false
threads: 0
threads_batch: 0
n_batch: 512
no_mmap: true
mlock: true
mul_mat_q: true
n_gpu_layers: 28
tensor_split: ''
n_ctx: 16384
llama_cpp_seed: 0.0
compress_pos_emb: 1
alpha_value: 1
rope_freq_base: 1000000
numa: true
|