metadata
language: en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- ruslanmv
- llama
- trl
- llama-3
- instruct
- finetune
- chatml
- DPO
- RLHF
- gpt4
- distillation
- heathcare
- medical
- clinical
- med
- lifescience
- Pharmaceutical
- Pharma
- TensorBlock
- GGUF
base_model: ruslanmv/Medical-Llama3-8B
datasets:
- ruslanmv/ai-medical-chatbot
widget:
- example_title: Medical-Llama3-8B
messages:
- role: system
content: >-
You are an expert and experienced from the healthcare and biomedical
domain with extensive medical knowledge and practical experience.
- role: user
content: How long does it take for newborn jaundice to go away?
output:
text: >-
Newborn jaundice, also known as neonatal jaundice, is a common condition
in newborns where the yellowing of the skin and eyes occurs due to an
elevated level of bilirubin in the blood. Bilirubin is a yellow pigment
that forms when red blood cells break down. In most cases, newborn
jaundice resolves on its own without any specific treatment. The
duration of newborn jaundice can vary depending on several factors such
as the underlying cause, gestational age at birth, and individual
variations in bilirubin metabolism. Here are some general guidelines
model-index:
- name: Medical-Llama3-8B
results: []
Feedback and support: TensorBlock's Twitter/X, Telegram Group and Discord server
ruslanmv/Medical-Llama3-8B - GGUF
This repo contains GGUF format model files for ruslanmv/Medical-Llama3-8B.
The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4011.
Prompt template
Model file specification
| Filename | Quant type | File Size | Description |
|---|---|---|---|
| Medical-Llama3-8B-Q2_K.gguf | Q2_K | 2.961 GB | smallest, significant quality loss - not recommended for most purposes |
| Medical-Llama3-8B-Q3_K_S.gguf | Q3_K_S | 3.413 GB | very small, high quality loss |
| Medical-Llama3-8B-Q3_K_M.gguf | Q3_K_M | 3.743 GB | very small, high quality loss |
| Medical-Llama3-8B-Q3_K_L.gguf | Q3_K_L | 4.025 GB | small, substantial quality loss |
| Medical-Llama3-8B-Q4_0.gguf | Q4_0 | 4.341 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
| Medical-Llama3-8B-Q4_K_S.gguf | Q4_K_S | 4.370 GB | small, greater quality loss |
| Medical-Llama3-8B-Q4_K_M.gguf | Q4_K_M | 4.583 GB | medium, balanced quality - recommended |
| Medical-Llama3-8B-Q5_0.gguf | Q5_0 | 5.215 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
| Medical-Llama3-8B-Q5_K_S.gguf | Q5_K_S | 5.215 GB | large, low quality loss - recommended |
| Medical-Llama3-8B-Q5_K_M.gguf | Q5_K_M | 5.339 GB | large, very low quality loss - recommended |
| Medical-Llama3-8B-Q6_K.gguf | Q6_K | 6.143 GB | very large, extremely low quality loss |
| Medical-Llama3-8B-Q8_0.gguf | Q8_0 | 7.954 GB | very large, extremely low quality loss - not recommended |
Downloading instruction
Command line
Firstly, install Huggingface Client
pip install -U "huggingface_hub[cli]"
Then, downoad the individual model file the a local directory
huggingface-cli download tensorblock/Medical-Llama3-8B-GGUF --include "Medical-Llama3-8B-Q2_K.gguf" --local-dir MY_LOCAL_DIR
If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:
huggingface-cli download tensorblock/Medical-Llama3-8B-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'