Update README.md

README.md

@@ -33,7 +33,7 @@ This repository provides a fine-tuned version of the powerful Llama3 8B model, s
 This model is accessible through the Hugging Face Transformers library. Install it using pip:
 
 ```bash
-pip install transformers
+pip install transformers   bitsandbytes  accelerate
 ```
 
 **Usage Example**
@@ -41,45 +41,51 @@ pip install transformers
 Here's a Python code snippet demonstrating how to interact with the `Medical-Llama3-8B-16bit` model and generate answers to your medical questions:
 
 ```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+import torch
+model_name = "ruslanmv/Medical-Llama3-8B"
+device_map = 'auto'
+bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",bnb_4bit_compute_dtype=torch.float16,)
+model = AutoModelForCausalLM.from_pretrained( model_name,quantization_config=bnb_config, trust_remote_code=True,use_cache=False,device_map=device_map)
+tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+tokenizer.pad_token = tokenizer.eos_token
 
-# Load tokenizer and model
-tokenizer = AutoTokenizer.from_pretrained("ruslanmv/Medical-Llama3-8B")
-model = AutoModelForCausalLM.from_pretrained("ruslanmv/Medical-Llama3-8B").to("cuda")  # If using GPU
-# Function to format and generate response with prompt engineering using a chat template
 def askme(question):
     sys_message = ''' 
     You are an AI Medical Assistant trained on a vast dataset of health information. Please be thorough and
     provide an informative answer. If you don't know the answer to a specific medical inquiry, advise seeking professional help.
-    '''
-    
+    '''   
     # Create messages structured for the chat template
     messages = [{"role": "system", "content": sys_message}, {"role": "user", "content": question}]
-
+    
     # Applying chat template
     prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
     inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
-    outputs = model.generate(**inputs, max_new_tokens=100, use_cache=True)  # Adjust max_new_tokens for longer responses
-
-    # Extract and return the generated text
-    answer = tokenizer.batch_decode(outputs)[0].strip()
+    outputs = model.generate(**inputs, max_new_tokens=100, use_cache=True)
+    
+    # Extract and return the generated text, removing the prompt
+    response_text = tokenizer.batch_decode(outputs)[0].strip()
+    answer = response_text.split('<|im_start|>assistant')[-1].strip()
     return answer
-
 # Example usage
 # - Context: First describe your problem.
 # - Question: Then make the question.
-question = '''
-I'm a 35-year-old male and for the past few months, I've been experiencing fatigue, increased sensitivity to cold, and dry, itchy skin.  
 
+question = '''I'm a 35-year-old male and for the past few months, I've been experiencing fatigue, 
+increased sensitivity to cold, and dry, itchy skin. 
 Could these symptoms be related to hypothyroidism? 
-If so, what steps should I take to get a proper diagnosis and discuss treatment options?
-'''
+If so, what steps should I take to get a proper diagnosis and discuss treatment options?'''
+
 print(askme(question))
+
 ```
 the type of answer is :
 ```
-Hello, I can understand your concern. I would suggest you to get a blood test done for TSH, T3, T4, and TPO antibodies. If the TSH is high, then you can be diagnosed with hypothyroidism. If the TSH is normal, then you can be diagnosed with Hashimoto's thyroiditis. In both cases, you will need to take levothyroxine. Hope I have answered your query. Let me know if I can assist you
-
+Based on your description, it sounds like you may be experiencing symptoms of hypothyroidism. 
+Hypothyroidism is a condition where the thyroid gland doesn't produce enough hormones, leading to a variety of symptoms. 
+Some common symptoms include fatigue, weight gain, constipation, and dry skin. 
+If you're experiencing any of these symptoms, it's important to see a doctor for a proper diagnosis and treatment plan. 
+Your doctor may order blood tests to check your thyroid hormone levels
 ```
 **Important Note**