|
--- |
|
library_name: transformers |
|
tags: |
|
- llama-cpp |
|
- gguf-my-repo |
|
- uncensored |
|
- transformers |
|
- llama |
|
- llama-3 |
|
- unsloth |
|
- llama-cpp |
|
- gguf-my-repo |
|
language: |
|
- en |
|
license: apache-2.0 |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
<div align="center"> |
|
<!-- Replace `#` with your actual links --> |
|
<a href="https://youtube.com/@devsdocode"><img alt="YouTube" src="https://img.shields.io/badge/YouTube-FF0000?style=for-the-badge&logo=youtube&logoColor=white"></a> |
|
<a href="https://t.me/devsdocode"><img alt="Telegram" src="https://img.shields.io/badge/Telegram-2CA5E0?style=for-the-badge&logo=telegram&logoColor=white"></a> |
|
<a href="https://www.instagram.com/sree.shades_/"><img alt="Instagram" src="https://img.shields.io/badge/Instagram-E4405F?style=for-the-badge&logo=instagram&logoColor=white"></a> |
|
<a href="https://www.linkedin.com/in/developer-sreejan/"><img alt="LinkedIn" src="https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white"></a> |
|
<a href="https://buymeacoffee.com/devsdocode"><img alt="Buy Me A Coffee" src="https://img.shields.io/badge/Buy%20Me%20A%20Coffee-FFDD00?style=for-the-badge&logo=buymeacoffee&logoColor=black"></a> |
|
</div> |
|
|
|
## Crafted with ❤️ by Devs Do Code (Sree) |
|
|
|
### GGUF Technical Specifications |
|
|
|
Delve into the intricacies of GGUF, a meticulously crafted format that builds upon the robust foundation of the GGJT model. Tailored for heightened extensibility and user-centric functionality, GGUF introduces a suite of indispensable features: |
|
|
|
**Single-file Deployment:** Streamline distribution and loading effortlessly. GGUF models have been meticulously architected for seamless deployment, necessitating no external files for supplementary information. |
|
|
|
**Extensibility:** Safeguard the future of your models. GGUF seamlessly accommodates the integration of new features into GGML-based executors, ensuring compatibility with existing models. |
|
|
|
**mmap Compatibility:** Prioritize efficiency. GGUF models are purposefully engineered to support mmap, facilitating rapid loading and saving, thus optimizing your workflow. |
|
|
|
**User-Friendly:** Simplify your coding endeavors. Load and save models effortlessly, irrespective of the programming language used, obviating the dependency on external libraries. |
|
|
|
**Full Information:** A comprehensive repository in a single file. GGUF models encapsulate all requisite information for loading, eliminating the need for users to furnish additional data. |
|
|
|
The differentiator between GGJT and GGUF lies in the deliberate adoption of a key-value structure for hyperparameters (now termed metadata). Bid farewell to untyped lists, and embrace a structured approach that seamlessly accommodates new metadata without compromising compatibility with existing models. Augment your model with supplementary information for enhanced inference and model identification. |
|
|
|
|
|
**QUANTIZATION_METHODS:** |
|
|
|
| Method | Quantization | Advantages | Trade-offs | |
|
|---|---|---|---| |
|
| q2_k | 2-bit integers | Significant model size reduction | Minimal impact on accuracy | |
|
| q3_k_l | 3-bit integers | Balance between model size reduction and accuracy preservation | Moderate impact on accuracy | |
|
| q3_k_m | 3-bit integers | Enhanced accuracy with mixed precision | Increased computational complexity | |
|
| q3_k_s | 3-bit integers | Improved model efficiency with structured pruning | Reduced accuracy | |
|
| q4_0 | 4-bit integers | Significant model size reduction | Moderate impact on accuracy | |
|
| q4_1 | 4-bit integers | Enhanced accuracy with mixed precision | Increased computational complexity | |
|
| q4_k_m | 4-bit integers | Optimized model size and accuracy with mixed precision and structured pruning | Reduced accuracy | |
|
| q4_k_s | 4-bit integers | Improved model efficiency with structured pruning | Reduced accuracy | |
|
| q5_0 | 5-bit integers | Balance between model size reduction and accuracy preservation | Moderate impact on accuracy | |
|
| q5_1 | 5-bit integers | Enhanced accuracy with mixed precision | Increased computational complexity | |
|
| q5_k_m | 5-bit integers | Optimized model size and accuracy with mixed precision and structured pruning | Reduced accuracy | |
|
| q5_k_s | 5-bit integers | Improved model efficiency with structured pruning | Reduced accuracy | |
|
| q6_k | 6-bit integers | Balance between model size reduction and accuracy preservation | Moderate impact on accuracy | |
|
| q8_0 | 8-bit integers | Significant model size reduction | Minimal impact on accuracy | |
|
|
|
|
|
|
|
<div align="center"> |
|
<!-- Replace `#` with your actual links --> |
|
<a href="https://youtube.com/@devsdocode"><img alt="YouTube" src="https://img.shields.io/badge/YouTube-FF0000?style=for-the-badge&logo=youtube&logoColor=white"></a> |
|
<a href="https://t.me/devsdocode"><img alt="Telegram" src="https://img.shields.io/badge/Telegram-2CA5E0?style=for-the-badge&logo=telegram&logoColor=white"></a> |
|
<a href="https://www.instagram.com/sree.shades_/"><img alt="Instagram" src="https://img.shields.io/badge/Instagram-E4405F?style=for-the-badge&logo=instagram&logoColor=white"></a> |
|
<a href="https://www.linkedin.com/in/developer-sreejan/"><img alt="LinkedIn" src="https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white"></a> |
|
<a href="https://buymeacoffee.com/devsdocode"><img alt="Buy Me A Coffee" src="https://img.shields.io/badge/Buy%20Me%20A%20Coffee-FFDD00?style=for-the-badge&logo=buymeacoffee&logoColor=black"></a> |
|
</div> |
