Harish2002 commited on
Commit
af4e7c6
Β·
verified Β·
1 Parent(s): 7a9d046

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +49 -22
README.md CHANGED
@@ -17,41 +17,68 @@ library_name: peft
17
  pipeline_tag: text-generation
18
  ---
19
 
20
- # CLI LoRA TinyLlama Fine-Tuning (Fenrir Internship)
21
 
22
- πŸš€ This model is a LoRA fine-tuned version of **TinyLlama-1.1B-Chat** on a custom dataset of command-line (CLI) Q&A. It was developed as part of a 24-hour AI/ML internship task by Fenrir Security Pvt Ltd.
 
 
23
 
24
  ## πŸ“ Dataset
25
- A carefully curated set of 200+ CLI Q&A pairs across tools like:
26
- - Git
27
- - Bash
28
- - `grep`, `tar`, `gzip`
29
- - `venv` and Python virtual environments
 
 
 
 
 
 
30
 
31
  ## βš™οΈ Model Details
32
- - **Base Model:** `TinyLlama-1.1B-Chat-v1.0`
33
- - **Fine-Tuning Method:** QLoRA via PEFT
34
- - **Hardware:** Local system (CPU or limited GPU)
35
- - **Epochs:** 3 (with early stopping)
36
- - **Tokenizer:** Inherited from base model
37
- - **Parameter Efficient:** ~7MB adapter weights only
 
 
 
 
 
38
 
39
  ## πŸ“Š Evaluation
40
- - Accuracy on known test Q&A: ~92%
41
- - Manual evaluation on unseen CLI inputs showed context-aware completions
42
- - Very low hallucination due to domain-specific training
 
 
 
 
 
 
 
 
43
 
44
  ## 🧠 Files Included
45
- - `adapter_model.safetensors`
46
- - `adapter_config.json`
47
- - `README.md` (you are here)
48
- - (Optional) `eval_logs.json`, `training.ipynb`
49
 
50
- ## πŸ“¦ Usage
 
 
 
 
 
 
 
 
 
 
51
 
52
  ```python
53
  from transformers import AutoTokenizer, AutoModelForCausalLM
54
- from peft import PeftModel, PeftConfig
55
 
56
  base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
57
  tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
 
17
  pipeline_tag: text-generation
18
  ---
19
 
20
+ # πŸ”§ CLI LoRA TinyLLaMA Fine-Tuning (Fenrir Internship Project)
21
 
22
+ πŸš€ This repository presents a **LoRA fine-tuned version of TinyLLaMA-1.1B-Chat** trained on a custom dataset of CLI Q&A. Developed as part of a 24-hour AI/ML internship task by **Fenrir Security Pvt Ltd**, this lightweight model functions as a domain-specific command-line assistant.
23
+
24
+ ---
25
 
26
  ## πŸ“ Dataset
27
+
28
+ A curated collection of 200+ real-world CLI Q&A pairs covering:
29
+
30
+ - Git (branching, stash, merge, rebase)
31
+ - Bash (variables, loops, file manipulation)
32
+ - `grep`, `tar`, `gzip` (command syntax, flags)
33
+ - Python environments (`venv`, pip)
34
+
35
+ Stored in `cli_questions.json`.
36
+
37
+ ---
38
 
39
  ## βš™οΈ Model Details
40
+
41
+ | Field | Value |
42
+ |-------------------|--------------------------------------------|
43
+ | Base Model | `TinyLlama/TinyLlama-1.1B-Chat-v1.0` |
44
+ | Fine-Tuning Method | QLoRA via `peft` |
45
+ | Epochs | 3 (with early stopping) |
46
+ | Adapter Size | ~7MB (LoRA weights only) |
47
+ | Hardware | Local CPU (low-resource) |
48
+ | Tokenizer | Inherited from base model |
49
+
50
+ ---
51
 
52
  ## πŸ“Š Evaluation
53
+
54
+ | Metric | Result |
55
+ |----------------------------|----------------|
56
+ | Accuracy on Eval Set | ~92% |
57
+ | Manual Review | High relevance |
58
+ | Hallucination Rate | Very low |
59
+ | Inference Time (CPU) | < 1s / query |
60
+
61
+ All results are stored in `eval_results.json`.
62
+
63
+ ---
64
 
65
  ## 🧠 Files Included
 
 
 
 
66
 
67
+ - `adapter_model.safetensors` β€” fine-tuned LoRA weights
68
+ - `adapter_config.json` β€” LoRA hyperparameters
69
+ - `training.ipynb` β€” complete training notebook
70
+ - `agent.py` β€” CLI interface to test the model
71
+ - `cli_questions.json` β€” training dataset
72
+ - `eval_results.json` β€” eval results
73
+ - `requirements.txt` β€” dependencies
74
+
75
+ ---
76
+
77
+ ## πŸ“¦ Inference Example
78
 
79
  ```python
80
  from transformers import AutoTokenizer, AutoModelForCausalLM
81
+ from peft import PeftModel
82
 
83
  base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
84
  tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")