File size: 7,315 Bytes

9faf67f

---
license: other
language:
- en
- fr
- es
- hi
- zh
- code
base_model: Locutusque/Orca-2-13b-SFT_v5
datasets:
- HuggingFaceH4/no_robots
- mlabonne/guanaco-llama2-1k
- OpenAssistant/oasst_top1_2023-08-25
- totally-not-an-llm/EverythingLM-data-V3
- garage-bAInd/Open-Platypus
widget:
- example_title: folklore
  text: '<|im_start|>user

    Are the stories of Nahanni Valley, like the decapitated men, true?

    <|im_end|>

    <|im_start|>assistant

    '
  output:
    text: 'The Nahanni Valley in Canada is a remote and rugged region that has long
      been shrouded in mystery. It is home to the South Nahanni River, which was designated
      as a UNESCO World Heritage Site due to its exceptional natural beauty and biodiversity.
      The valley''s history includes tales of gold prospectors, fur traders, and indigenous
      populations, but it also features some unexplained events and legends. One such
      story involves two prospectors who allegedly encountered an unknown creature
      that led them into the forest before returning without their heads. This tale
      has been passed down through generations, but there are no definitive answers
      regarding its veracity.


      It is important to note that these stories may have originated from local folklore
      or were exaggerated over time. There is no concrete evidence to support the
      existence of supernatural creatures or paranormal occurrences within the Nahanni
      Valley. Nonetheless, the area remains a captivating destination for those seeking
      adventure and exploration. If you have any questions about this topic or need
      assistance related to traveling to the Nahanni Valley, please feel free to ask!'
- example_title: Alan Turing
  text: '<|im_start|>user

    Why was Alan Turing castrated?

    <|im_end|>

    <|im_start|>assistant

    '
  output:
    text: 'Alan Turing, a renowned mathematician and computer scientist who played
      a crucial role in breaking the German Enigma code during World War II, faced
      significant challenges due to his homosexuality. In 1952, he was convicted of
      "gross indecency" for engaging in a consensual relationship with another man,
      which led to him being sentenced to hormone therapy designed to reduce his sex
      drive. This treatment included the administration of estrogen, which had the
      side effect of causing breast growth.


      In an effort to avoid this unwanted physical change, Turing requested that his
      doctors cease the hormone treatments. However, they ignored his request and
      continued the therapy without his consent. As a result, Turing''s breasts began
      to grow, leading him to undergo surgery to remove them. The operation, known
      as a mastectomy, caused considerable pain and discomfort, further contributing
      to his emotional turmoil.


      It is important to note that Turing''s castration was not directly carried out
      by the authorities but rather resulted from the unintended consequences of the
      forced hormone therapy.'
tags:
- TensorBlock
- GGUF
---

<div style="width: auto; margin-left: auto; margin-right: auto">
<img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
</div>
<div style="display: flex; justify-content: space-between; width: 100%;">
    <div style="display: flex; flex-direction: column; align-items: flex-start;">
        <p style="margin-top: 0.5em; margin-bottom: 0em;">
            Feedback and support: TensorBlock's  <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
        </p>
    </div>
</div>

## Locutusque/Orca-2-13b-SFT_v5 - GGUF

This repo contains GGUF format model files for [Locutusque/Orca-2-13b-SFT_v5](https://huggingface.co/Locutusque/Orca-2-13b-SFT_v5).

The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).

## Prompt template

```

```

## Model file specification

| Filename | Quant type | File Size | Description |
| -------- | ---------- | --------- | ----------- |
| [Orca-2-13b-SFT_v5-Q2_K.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q2_K.gguf) | Q2_K | 4.521 GB | smallest, significant quality loss - not recommended for most purposes |
| [Orca-2-13b-SFT_v5-Q3_K_S.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q3_K_S.gguf) | Q3_K_S | 5.270 GB | very small, high quality loss |
| [Orca-2-13b-SFT_v5-Q3_K_M.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q3_K_M.gguf) | Q3_K_M | 5.903 GB | very small, high quality loss |
| [Orca-2-13b-SFT_v5-Q3_K_L.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q3_K_L.gguf) | Q3_K_L | 6.454 GB | small, substantial quality loss |
| [Orca-2-13b-SFT_v5-Q4_0.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q4_0.gguf) | Q4_0 | 6.860 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
| [Orca-2-13b-SFT_v5-Q4_K_S.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q4_K_S.gguf) | Q4_K_S | 6.913 GB | small, greater quality loss |
| [Orca-2-13b-SFT_v5-Q4_K_M.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q4_K_M.gguf) | Q4_K_M | 7.326 GB | medium, balanced quality - recommended |
| [Orca-2-13b-SFT_v5-Q5_0.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q5_0.gguf) | Q5_0 | 8.356 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
| [Orca-2-13b-SFT_v5-Q5_K_S.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q5_K_S.gguf) | Q5_K_S | 8.356 GB | large, low quality loss - recommended |
| [Orca-2-13b-SFT_v5-Q5_K_M.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q5_K_M.gguf) | Q5_K_M | 8.596 GB | large, very low quality loss - recommended |
| [Orca-2-13b-SFT_v5-Q6_K.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q6_K.gguf) | Q6_K | 9.946 GB | very large, extremely low quality loss |
| [Orca-2-13b-SFT_v5-Q8_0.gguf](https://huggingface.co/tensorblock/Orca-2-13b-SFT_v5-GGUF/tree/main/Orca-2-13b-SFT_v5-Q8_0.gguf) | Q8_0 | 12.881 GB | very large, extremely low quality loss - not recommended |


## Downloading instruction

### Command line

Firstly, install Huggingface Client

```shell
pip install -U "huggingface_hub[cli]"
```

Then, downoad the individual model file the a local directory

```shell
huggingface-cli download tensorblock/Orca-2-13b-SFT_v5-GGUF --include "Orca-2-13b-SFT_v5-Q2_K.gguf" --local-dir MY_LOCAL_DIR
```

If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:

```shell
huggingface-cli download tensorblock/Orca-2-13b-SFT_v5-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
```