RichardErkhov/nisten_-_BigCodeLlama-92b-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

BigCodeLlama-92b - GGUF

• Model creator: https://huggingface.co/nisten/
• Original model: https://huggingface.co/nisten/BigCodeLlama-92b/

Name	Quant method	Size
BigCodeLlama-92b.Q2_K.gguf	Q2_K	31.52GB
BigCodeLlama-92b.IQ3_XS.gguf	IQ3_XS	1.11GB
BigCodeLlama-92b.IQ3_S.gguf	IQ3_S	15.38GB
BigCodeLlama-92b.Q3_K_S.gguf	Q3_K_S	36.95GB
BigCodeLlama-92b.IQ3_M.gguf	IQ3_M	33.92GB
BigCodeLlama-92b.Q3_K.gguf	Q3_K	41.22GB
BigCodeLlama-92b.Q3_K_M.gguf	Q3_K_M	41.22GB
BigCodeLlama-92b.Q3_K_L.gguf	Q3_K_L	44.92GB
BigCodeLlama-92b.IQ4_XS.gguf	IQ4_XS	46.21GB
BigCodeLlama-92b.Q4_0.gguf	Q4_0	48.31GB
BigCodeLlama-92b.IQ4_NL.gguf	IQ4_NL	48.77GB
BigCodeLlama-92b.Q4_K_S.gguf	Q4_K_S	48.67GB
BigCodeLlama-92b.Q4_K.gguf	Q4_K	51.4GB
BigCodeLlama-92b.Q4_K_M.gguf	Q4_K_M	51.4GB
BigCodeLlama-92b.Q4_1.gguf	Q4_1	53.65GB
BigCodeLlama-92b.Q5_0.gguf	Q5_0	59.0GB
BigCodeLlama-92b.Q5_K_S.gguf	Q5_K_S	59.0GB
BigCodeLlama-92b.Q5_K.gguf	Q5_K	60.59GB
BigCodeLlama-92b.Q5_K_M.gguf	Q5_K_M	60.59GB
BigCodeLlama-92b.Q5_1.gguf	Q5_1	64.34GB
BigCodeLlama-92b.Q6_K.gguf	Q6_K	70.35GB
BigCodeLlama-92b.Q8_0.gguf	Q8_0	91.12GB

Original model description:

base_model: [codellama/CodeLlama-70b-Instruct-hf] tags: - mergekit - merge - code license: mit pipeline_tag: conversational

BigCodeLLama 92b LFG 🚀

Experimental 92B CodeLlaMA frankenstein to see how it benchmarks

Models Merged with base codellama/CodeLlama-70b-Instruct-hf

Models Merged

The following models were included in the merge:

• ../CodeLlama-70b-Python-hf
• ../CodeLlama-70b-Instruct-hf

Configuration

The following YAML configuration was used to produce this model:

dtype: bfloat16
merge_method: passthrough
slices:
- sources:
  - layer_range: [0, 69]
    model:
      model:
        path: ../CodeLlama-70b-Instruct-hf
- sources:
  - layer_range: [42, 80]
    model:
      model:
        path: ../CodeLlama-70b-Python-hf

Gguf available here https://huggingface.co/nisten/BigCodeLlama-92b-GGUF