File size: 4,146 Bytes
fb4bfd8 c05f5a3 edd762c dc1122f b00cf22 dc1122f 865c0cb 1bf945b dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 b651ec8 dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 dc1122f b00cf22 dc1122f a1d0ce3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
---
datasets:
- brucewayne0459/Skin_diseases_and_care
language:
- en
license: mit
tags:
- medical
- dermatology
- skin_disease
- skin_care
- unsloth
- trl
- sft
---
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
- **Developed by:** Bruce_Wayne(The Batman)
- **Funded by [optional]:** Wayne Industies
- **Model type:** Text Generation
- **Finetuned from model [optional]:** OpenBioLLM(llama-3)(aaditya/Llama3-OpenBioLLM-8B)
## You can find the gguf versions here --> https://huggingface.co/brucewayne0459/OpenBioLLm-Derm-gguf
### please let me know how the model works -->https://forms.gle/N14zZTkLpUr6Hf4BA
### Thank you!
## Uses
### Direct Use
This model is fine-tuned on skin diseases and dermatology data and is used for a dermatology chatbot to provide clear, accurate, and helpful information about various skin diseases, skin care routines, treatments, and related dermatological advice.
## Bias, Risks, and Limitations
This model is trained on dermatology data, which might contain inherent biases. It is important to note that the model's responses should not be considered a substitute for professional medical advice. There may be limitations in understanding rare skin conditions or those not well-represented in the training data.
The model still need to be fine-tuned further to get accurate answers.
### Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
## How to Get Started with the Model
Use the code below to get started with the model.
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "brucewayne0459/OpenBioLLm-Derm"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
```
## Training Details
### Training Data
The model is fine-tuned on a dataset containing information about various skin diseases and dermatology care.
brucewayne0459/Skin_diseases_and_care
### Training Procedure
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
#### Preprocessing [optional]
"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
You are a highly knowledgeable and empathetic dermatologist. Provide clear, accurate, and helpful information about various skin diseases, skin care routines, treatments, and related dermatological advice.
### Input:
{}
### Response:
{}
"""
EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN
def formatting_prompts_func(examples):
inputs = examples["Topic"]
outputs = examples["Information"]
texts = []
Prompt passed while fine tuning the model
#### Training Hyperparameters
Training regime: The model was trained using the following hyperparameters:
Per device train batch size: 2
Gradient accumulation steps: 4
Warmup steps: 5
Max steps: 120
Learning rate: 2e-4
Optimizer: AdamW (8-bit)
Weight decay: 0.01
LR scheduler type: Linear
## Environmental Impact
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
- **Hardware Type:** Tesls T4 gpu
- **Hours used:** 1hr
- **Cloud Provider:** Google Colab
## Technical Specifications [optional]
### Model Architecture and Objective
This model is based on the LLaMA (Large Language Model Meta AI) architecture and fine-tuned to provide dermatological advice.
#### Hardware
The training was performed on Tesla T4 gpu with 4-bit quantization and gradient checkpointing to optimize memory usage.
### Feel free to provide any missing details or correct the assumptions made, and I'll update the model card accordingly.
|