Create README.md

README.md

@@ -0,0 +1,108 @@
+---
+license: apache-2.0
+datasets:
+- NIEXCHE/turkish_agriculture_QA_llama2_22.6k
+language:
+- tr
+- en
+---
+
+# Model Card for LLaMA-2-7B-NIEXCHE
+
+This model was fine-tuned from LLaMA-2-7B on a Turkish agriculture QA dataset. It supports both Turkish and English languages and was trained for use in agriculture-related natural language processing (NLP) tasks.
+
+## Model Details
+
+### Model Description
+
+- **Developed by:** NIEXCHE (Fevzi KILAS)
+- **Finetuned from model:** [meta-llama/Llama-2-7b](https://huggingface.co/meta-llama/Llama-2-7b)
+- **License:** Apache-2.0
+- **Language(s) (NLP):** Turkish, English
+- **Model type:** LLaMA-2-based model
+- **Training Dataset:** [NIEXCHE/turkish_agriculture_QA_llama2_22.6k](https://huggingface.co/datasets/NIEXCHE/turkish_agriculture_QA_llama2_22.6k)
+
+### Model Sources
+
+- **Repository:** [Model Repository (TBA)](#)
+- **Demo:** [TBA](#)
+
+## Uses
+
+### Direct Use
+
+The model can be used directly for question-answering tasks related to agriculture in Turkish and English. It is fine-tuned specifically for agricultural Q&A, making it suitable for similar domains and use cases.
+
+### Out-of-Scope Use
+
+The model might not perform well on general knowledge questions outside of the agriculture domain.
+
+## Training Details
+
+### Training Data
+
+The training data was a custom dataset created by translating and cleaning agricultural QA data from [this source](https://huggingface.co/datasets/KisanVaani/agriculture-qa-english-only). The dataset contains 22.6k question-answer pairs in Turkish.
+
+### Training Procedure
+
+The model was trained using the following frameworks and libraries:
+- **Frameworks:** PyTorch, `transformers`, `accelerate==0.21.0`, `peft==0.4.0`, `bitsandbytes==0.40.2`, `trl==0.4.7`
+- **Precision:** The model was trained using 4-bit quantization (BNB) with mixed precision (`float16`) to optimize memory usage.
+
+#### Training Hyperparameters
+
+- **Base Model:** `meta-llama/Llama-2-7b`
+- **Batch Size:** 4 (per device)
+- **Learning Rate:** 2e-4
+- **LoRA Parameters:** 
+  - lora_r = 64
+  - lora_alpha = 16
+  - lora_dropout = 0.1
+- **Epochs:** 1
+- **Optimizer:** Paged AdamW (32-bit)
+- **Gradient Accumulation Steps:** 1
+- **Scheduler:** Cosine
+- **Max Gradient Norm:** 0.3
+- **Gradient Checkpointing:** Enabled
+- **Warmup Ratio:** 0.03
+- **Group by Length:** Enabled
+- **Max Sequence Length:** None
+
+### Hardware
+
+- **Training Hardware:** Google Colab Pro (A100 GPU) and 53 GB system RAM.
+- **Training Time:** Approximately 1 hour 40 minutes.
+
+Training output:
+`TrainOutput(global_step=5654, training_loss=0.7829279924898043, metrics={'train_runtime': 6029.996, 'train_samples_per_second': 3.75, 'train_steps_per_second': 0.938, 'total_flos': 5.516196145999872e+16, 'train_loss': 0.7829279924898043, 'epoch': 1.0})`
+
+
+
+## Evaluation
+
+The same dataset (`NIEXCHE/turkish_agriculture_QA_llama2_22.6k`) was used for evaluation purposes.
+
+## Environmental Impact
+
+Carbon emissions were estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute).
+
+- **Hardware Type:** Google Colab (A100 GPU)
+- **Hours used:** 1 hour 40 minutes
+- **Compute Region:** Google Cloud (Colab)
+- **Carbon Emitted:** Estimations pending
+
+## Citation
+
+If you use this model in your research or applications, please cite it as:
+
+```bibtex
+@misc{Fevzi2024LLaMA-2-7B-NIEXCHE,
+  author = {Fevzi KILAS},
+  title = {LLaMA-2-7B-NIEXCHE: A Turkish Agriculture QA Model},
+  year = {2024},
+  howpublished = {https://huggingface.co/NIEXCHE/turkish_agriculture_QA_llama2_22.6k}
+}
+```
+## Contact:
+
+[NIEXCHE (Fevzi KILAS)](https://niexche.github.io/)