metadata
license: mit
datasets:
- keivalya/MedQuad-MedicalQnADataset
language:
- en
library_name: peft
tags:
- medical
pipeline_tag: question-answering
Model Card for GaiaMiniMed
This is a medical fine tuned model from the Falcon-7b-Instruction Base using 500 steps & 6 epochs with MedAware Dataset from keivalya
Model Details
Model Description
- Developed by: Tonic
- Shared by : Tonic
- Model type: Medical Fine-Tuned Conversational Falcon 7b (Instruct)
- Language(s) (NLP): English
- License: MIT
- Finetuned from model:tiiuae/falcon-7b-instruct
Model Sources
- Repository: Github
- Demo : pseudolab/gaiaminimed & tonic/gaiaminimed
Uses
Use this model like you would use Falcon Instruct Models
Direct Use
This model is intended for educational purposes only , always consult a doctor for the best advice.
This model should perform better at medical QnA tasks in a conversational manner.
It is our hope that it will help improve patient outcomes and public health.
Downstream Use
Use this model next to others and have group conversations to produce diagnoses , public health advisory , and personal hygene improvements.
Out-of-Scope Use
This model is not meant as a decision support system in the wild, only for educational use.
Bias, Risks, and Limitations
{{ bias_risks_limitations | default("[More Information Needed]", true)}}
How to Get Started with the Model
Try it here : Pseudolab/GaiaMiniMed
See the author's demo
Use the code below to get started with the model.
from transformers import AutoConfig, AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig
import torch
import gradio as gr
# Define the device
device = "cuda" if torch.cuda.is_available() else "cpu"
# Use model IDs as variables
base_model_id = "tiiuae/falcon-7b-instruct"
model_directory = "Tonic/GaiaMiniMed"
# Instantiate the Tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True, padding_side="left")
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = 'left'
# Define the PeftConfig
#peft_config = PeftConfig(
# max_length=500,
# use_cache=True,
# early_stopping=False,
# bos_token_id=tokenizer.bos_token_id, # Use the tokenizer's BOS token ID
# eos_token_id=tokenizer.eos_token_id, # Use the tokenizer's EOS token ID
# pad_token_id=tokenizer.eos_token_id, # Use the tokenizer's EOS token ID
# temperature=0.4,
# do_sample=True
#)
# Load the GaiaMiniMed model with the specified configuration
# Load the Peft model with a specific configuration
# Specify the configuration class for the model
model_config = PeftConfig.from_pretrained(model_directory) #use base falcon config
# Load the PEFT model with the specified configuration
peft_model = AutoModelForCausalLM.from_pretrained(base_model_id, config=model_config)
peft_model = PeftModel.from_pretrained(model="Tonic/GaiaMiniMed")
peft_model = PeftModel.from_pretrained(peft_model, "Tonic/GaiaMiniMed")
# Class to encapsulate the Falcon chatbot
class FalconChatBot:
def __init__(self, system_prompt="You are an expert medical analyst:"):
self.system_prompt = system_prompt
def process_history(self, history):
# Filter out special commands from the history
filtered_history = []
for message in history:
user_message = message["user"]
assistant_message = message["assistant"]
# Check if the user_message is not a special command
if not user_message.startswith("Falcon:"):
filtered_history.append({"user": user_message, "assistant": assistant_message})
return filtered_history
def predict(self, input_data, max_length=500):
# Extract messages from the input data
preprompt = input_data["preprompt"]
history = input_data["history"]
# Process the history to remove special commands
processed_history = self.process_history(history)
# Generate the formatted conversation in Falcon message format
conversation = f"{preprompt}\n"
for message in processed_history:
user_message = message["user"]
assistant_message = message["assistant"]
conversation += f"Falcon:{' ' + assistant_message if assistant_message else ''} User: {user_message}\n Falcon:\n"
# Encode the formatted conversation using the tokenizer
input_ids = tokenizer.encode(conversation, return_tensors="pt", add_special_tokens=False)
# Generate a response using the Falcon model
response = falcon_model.generate(input_ids, max_length=max_length, use_cache=True, early_stopping=True, bos_token_id=falcon_model.config.bos_token_id, eos_token_id=falcon_model.config.eos_token_id, pad_token_id=falcon_model.config.eos_token_id, temperature=0.4, do_sample=True)
# Decode the generated response to text
response_text = tokenizer.decode(response[0], skip_special_tokens=True)
return response_text
# Create the Falcon chatbot instance
falcon_bot = FalconChatBot()
# Define the Gradio interface
title = "馃憢馃徎Welcome to Tonic's 馃Falcon's Medical馃懆馃徎鈥嶁殨锔廍xpert Chat馃殌"
description = "You can use this Space to test out the GaiaMiniMed model [(Tonic/GaiaMiniMed)](https://huggingface.co/Tonic/GaiaMiniMed) or duplicate this Space and use it locally or on 馃HuggingFace. [Join me on Discord to build together](https://discord.gg/VqTxc76K3u)."
examples = [{
"preprompt": "system message",
"history": [{
"user": "user message 1",
"assistant": "assistant message 1"
}, {
"user": "user message 1",
"assistant": None
}]
}]
iface = gr.Interface(
fn=falcon_bot.predict,
title=title,
description=description,
examples=examples,
inputs=[
gr.inputs.Textbox(label="Input Data", type="json"),
],
outputs="text",
theme="ParityError/Anime"
)
# Launch the Gradio interface for the Falcon model
iface.launch()
Training Details
Results
TrainOutput(global_step=6150, training_loss=1.0597990553941183,
{'epoch': 6.0})
Training Data
DatasetDict({
train: Dataset({
features: ['qtype', 'Question', 'Answer'],
num_rows: 16407
})
})
Training Procedure
Preprocessing [optional]
trainable params: 4718592 || all params: 3613463424 || trainables%: 0.13058363808693696
Training Hyperparameters
- Training regime: {{ training_regime | default("[More Information Needed]", true)}}
Speeds, Sizes, Times [optional]
metrics={'train_runtime': 30766.4612, 'train_samples_per_second': 3.2, 'train_steps_per_second': 0.2,
'total_flos': 1.1252790565109983e+18, 'train_loss': 1.0597990553941183,", true)}}
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: {{ hardware | default("[More Information Needed]", true)}}
- Hours used: {{ hours_used | default("[More Information Needed]", true)}}
- Cloud Provider: {{ cloud_provider | default("[More Information Needed]", true)}}
- Compute Region: {{ cloud_region | default("[More Information Needed]", true)}}
- Carbon Emitted: {{ co2_emitted | default("[More Information Needed]", true)}}
Technical Specifications
Model Architecture and Objective
PeftModelForCausalLM(
(base_model): LoraModel(
(model): FalconForCausalLM(
(transformer): FalconModel(
(word_embeddings): Embedding(65024, 4544)
(h): ModuleList(
(0-31): 32 x FalconDecoderLayer(
(self_attention): FalconAttention(
(maybe_rotary): FalconRotaryEmbedding()
(query_key_value): Linear4bit(
in_features=4544, out_features=4672, bias=False
(lora_dropout): ModuleDict(
(default): Dropout(p=0.05, inplace=False)
)
(lora_A): ModuleDict(
(default): Linear(in_features=4544, out_features=16, bias=False)
)
(lora_B): ModuleDict(
(default): Linear(in_features=16, out_features=4672, bias=False)
)
(lora_embedding_A): ParameterDict()
(lora_embedding_B): ParameterDict()
)
(dense): Linear4bit(in_features=4544, out_features=4544, bias=False)
(attention_dropout): Dropout(p=0.0, inplace=False)
)
(mlp): FalconMLP(
(dense_h_to_4h): Linear4bit(in_features=4544, out_features=18176, bias=False)
(act): GELU(approximate='none')
(dense_4h_to_h): Linear4bit(in_features=18176, out_features=4544, bias=False)
)
(input_layernorm): LayerNorm((4544,), eps=1e-05, elementwise_affine=True)
)
)
(ln_f): LayerNorm((4544,), eps=1e-05, elementwise_affine=True)
)
(lm_head): Linear(in_features=4544, out_features=65024, bias=False)
)
)
)
Compute Infrastructure
Google Collaboratory
Hardware
A100