Update README.md

README.md

@@ -10,17 +10,10 @@ base_model:
 ---
 
 # Model Card for Model ID
-
 <!-- Provide a quick summary of what the model is/does. -->
-
-
-
 ## Model Details
-
 ### Model Description
-
 <!-- Provide a longer summary of what this model is. -->
-
 **Fine-Tuned Llama 3.1 3B Instruct with Medical Terms using QLoRA**
 
 This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
@@ -39,17 +32,7 @@ The fine-tuning process involves using **QLoRA** to adapt the pre-trained model
 - **Quantization:** 4-bit NF4 (Normal Float 4) Quantization
 - **Hardware Used:** Consumer-grade GPU with 4-bit memory optimization
 
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-
 ## How to Get Started with the Model
-
 Use the code below to get started with the model.
 
 ```python
@@ -69,25 +52,21 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
 
 ## Training Details
-
 ### Training Data
-
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
 The model has been fine-tuned on the **dmedhi/wiki_medical_terms** dataset. This dataset is designed to improve medical terminology comprehension and consists of:
 
 ✅ Medical definitions and terminologies
+
 ✅ Disease symptoms and conditions
+
 ✅ Healthcare and clinical knowledge from Wikipedia's medical section
 
 This dataset ensures that the fine-tuned model performs well in understanding and responding to medical queries with enhanced accuracy.
 
 ### Training Procedure
-
 <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
 #### Preprocessing
-
 - The dataset was cleaned and tokenized using the Llama 3.1 tokenizer, ensuring that medical terms were preserved.
 
 - Special medical terminologies were handled properly to maintain context.
@@ -96,9 +75,6 @@ This dataset ensures that the fine-tuned model performs well in understanding an
 
 
 #### Training Hyperparameters
-
-- **Training regime:**  <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
 - **Training regime:** bf16 mixed precision (to balance efficiency and precision)
 - **Batch Size:** 1 per device
 - **Gradient Accumulation Steps:** 4 (to simulate a larger batch size)
@@ -111,9 +87,7 @@ This dataset ensures that the fine-tuned model performs well in understanding an
 - **LoRA Dropout:** 0.05
 
 #### Speeds, Sizes, Times
-
 <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
 - **Training Hardware:** Single GPU (consumer-grade, VRAM-optimized)
 - **Model Size after Fine-Tuning:** Approx. 3B parameters with LoRA adapters
 - **Training Time:** ~3-4 hours per epoch on A100 40GB GPU
@@ -121,9 +95,7 @@ This dataset ensures that the fine-tuned model performs well in understanding an
 
 
 ## Environmental Impact
-
 <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
 Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
 
 - **Hardware Type:** A100 40 GB GPU
@@ -133,25 +105,23 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 - **Carbon Emitted:** [More Information Needed]
 
 ## Limitations & Considerations
-
 ❗ Not a substitute for professional medical advice
 ❗ May contain biases from training data
 ❗ Limited knowledge scope (not updated in real-time)
 
 ## Citation
-
 <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
 If you use this model, please consider citing:
-
+```bibtex
 @article{llama3.1_medical_qlora,
   title={Fine-tuned Llama 3.1 3B Instruct for Medical Knowledge with QLoRA},
   author={Karthik Manjunath Hadagali},
   year={2024},
   journal={Hugging Face Model Repository}
 }
+```
 
 ## Acknowledgments
-
 - Meta AI for the Llama 3.1 3B Instruct Model.
 - Hugging Face PEFT for QLoRA implementation.
 - dmedhi/wiki_medical_terms dataset contributors.