|
--- |
|
datasets: |
|
- brucewayne0459/Skin_diseases_and_care |
|
language: |
|
- en |
|
license: mit |
|
tags: |
|
- medical |
|
- dermatology |
|
- skin_disease |
|
- skin_care |
|
- unsloth |
|
- trl |
|
- sft |
|
--- |
|
# Model Card for Model ID |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1). |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
|
|
|
|
- **Developed by:** Bruce_Wayne(The Batman) |
|
- **Funded by [optional]:** Wayne Industies |
|
- **Model type:** Text Generation |
|
- **Finetuned from model [optional]:** OpenBioLLM(llama-3)(aaditya/Llama3-OpenBioLLM-8B) |
|
|
|
|
|
## Uses |
|
|
|
### Direct Use |
|
|
|
This model is fine-tuned on skin diseases and dermatology data and is used for a dermatology chatbot to provide clear, accurate, and helpful information about various skin diseases, skin care routines, treatments, and related dermatological advice. |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
|
This model is trained on dermatology data, which might contain inherent biases. It is important to note that the model's responses should not be considered a substitute for professional medical advice. There may be limitations in understanding rare skin conditions or those not well-represented in the training data. |
|
The model still need to be fine-tuned further to get accurate answers. |
|
|
|
### Recommendations |
|
|
|
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. |
|
|
|
## How to Get Started with the Model |
|
|
|
Use the code below to get started with the model. |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
model_name = "brucewayne0459/OpenBioLLm-Derm" |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
model = AutoModelForCausalLM.from_pretrained(model_name) |
|
``` |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
The model is fine-tuned on a dataset containing information about various skin diseases and dermatology care. |
|
brucewayne0459/Skin_diseases_and_care |
|
|
|
### Training Procedure |
|
|
|
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> |
|
|
|
#### Preprocessing [optional] |
|
|
|
"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. |
|
|
|
### Instruction: |
|
You are a highly knowledgeable and empathetic dermatologist. Provide clear, accurate, and helpful information about various skin diseases, skin care routines, treatments, and related dermatological advice. |
|
|
|
### Input: |
|
{} |
|
|
|
### Response: |
|
{} |
|
""" |
|
EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN |
|
|
|
def formatting_prompts_func(examples): |
|
inputs = examples["Topic"] |
|
outputs = examples["Information"] |
|
texts = [] |
|
|
|
Prompt passed while fine tuning the model |
|
#### Training Hyperparameters |
|
|
|
Training regime: The model was trained using the following hyperparameters: |
|
Per device train batch size: 2 |
|
Gradient accumulation steps: 4 |
|
Warmup steps: 5 |
|
Max steps: 120 |
|
Learning rate: 2e-4 |
|
Optimizer: AdamW (8-bit) |
|
Weight decay: 0.01 |
|
LR scheduler type: Linear |
|
|
|
|
|
|
|
## Environmental Impact |
|
|
|
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly --> |
|
|
|
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). |
|
|
|
- **Hardware Type:** Tesls T4 gpu |
|
- **Hours used:** 1hr |
|
- **Cloud Provider:** Google Colab |
|
|
|
|
|
## Technical Specifications [optional] |
|
|
|
### Model Architecture and Objective |
|
|
|
This model is based on the LLaMA (Large Language Model Meta AI) architecture and fine-tuned to provide dermatological advice. |
|
|
|
|
|
#### Hardware |
|
|
|
The training was performed on Tesla T4 gpu with 4-bit quantization and gradient checkpointing to optimize memory usage. |
|
|
|
|
