Update README.md

README.md

@@ -33,26 +33,40 @@ The model is not designed for complex sentence structures, idiomatic expressions
 
 Users (both direct and downstream) should be aware that the model's accuracy may decline with more complex or less conventional sentence structures. It's recommended to use this model in conjunction with other tools for more comprehensive linguistic analysis.
 
+
 ## Training Details
 
+The model was trained on a curated dataset of simple English sentences annotated with Universal Dependency Parsing tags. The dataset was sourced from the "manupinasco/syntax_analysis" dataset available on Hugging Face's Datasets Hub. The training data focused on ensuring high accuracy in syntactic role assignment, aiming to improve the model's ability to understand and generate syntactically correct responses.
+
 ### Training Data
 
 The model was trained on a curated dataset of simple English sentences annotated with Universal Dependency Parsing tags. The training data focused on ensuring high accuracy in syntactic role assignment.
 
 ### Training Procedure
 
+The training procedure involved fine-tuning the unsloth/Meta-Llama-3.1-8B-Instruct model using a custom prompt format inspired by the Alpaca prompt template. The procedure included quantization to 4-bit to reduce memory usage, and mixed precision training to leverage GPU capabilities effectively.
 
+Key components of the training process:
 
-#### Training Hyperparameters
+Model Quantization: 4-bit quantization was applied to the model to reduce VRAM usage while maintaining performance.
+Gradient Checkpointing: Enabled using "unsloth" mode to save memory during training, which allowed handling longer sequences effectively.
+Prompt Template: The model was trained using a structured prompt that provided instructions and expected responses, ensuring consistency and clarity in the tasks presented to the model.
 
-- **Training regime:** Mixed precision (fp16)
+#### Training Hyperparameters
 
+- **Batch Size**: 2 per device
+- **Gradient Accumulation**: 4 steps
+- **Warmup Steps**: 5
+- **Max Training Steps**: 60
+- **Learning Rate**: 2e-4
+- **Optimizer**: AdamW with 8-bit quantization
+- **Weight Decay**: 0.01
+- **LR Scheduler**: Linear
+- **Mixed Precision**: fp16 (or bf16 if supported)
+- 
 ## Evaluation
 
-
-
-### Testing Data, Factors & Metrics
-
+The model's performance was evaluated using the test split from the same dataset. The evaluation focused on syntactic role assignment accuracy.
 
 #### Testing Data