Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -17,41 +17,68 @@ library_name: peft
|
|
17 |
pipeline_tag: text-generation
|
18 |
---
|
19 |
|
20 |
-
# CLI LoRA
|
21 |
|
22 |
-
π This
|
|
|
|
|
23 |
|
24 |
## π Dataset
|
25 |
-
|
26 |
-
-
|
27 |
-
|
28 |
-
-
|
29 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
30 |
|
31 |
## βοΈ Model Details
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
-
|
37 |
-
|
|
|
|
|
|
|
|
|
|
|
38 |
|
39 |
## π Evaluation
|
40 |
-
|
41 |
-
|
42 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
43 |
|
44 |
## π§ Files Included
|
45 |
-
- `adapter_model.safetensors`
|
46 |
-
- `adapter_config.json`
|
47 |
-
- `README.md` (you are here)
|
48 |
-
- (Optional) `eval_logs.json`, `training.ipynb`
|
49 |
|
50 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
51 |
|
52 |
```python
|
53 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
54 |
-
from peft import PeftModel
|
55 |
|
56 |
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
|
57 |
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
|
|
|
17 |
pipeline_tag: text-generation
|
18 |
---
|
19 |
|
20 |
+
# π§ CLI LoRA TinyLLaMA Fine-Tuning (Fenrir Internship Project)
|
21 |
|
22 |
+
π This repository presents a **LoRA fine-tuned version of TinyLLaMA-1.1B-Chat** trained on a custom dataset of CLI Q&A. Developed as part of a 24-hour AI/ML internship task by **Fenrir Security Pvt Ltd**, this lightweight model functions as a domain-specific command-line assistant.
|
23 |
+
|
24 |
+
---
|
25 |
|
26 |
## π Dataset
|
27 |
+
|
28 |
+
A curated collection of 200+ real-world CLI Q&A pairs covering:
|
29 |
+
|
30 |
+
- Git (branching, stash, merge, rebase)
|
31 |
+
- Bash (variables, loops, file manipulation)
|
32 |
+
- `grep`, `tar`, `gzip` (command syntax, flags)
|
33 |
+
- Python environments (`venv`, pip)
|
34 |
+
|
35 |
+
Stored in `cli_questions.json`.
|
36 |
+
|
37 |
+
---
|
38 |
|
39 |
## βοΈ Model Details
|
40 |
+
|
41 |
+
| Field | Value |
|
42 |
+
|-------------------|--------------------------------------------|
|
43 |
+
| Base Model | `TinyLlama/TinyLlama-1.1B-Chat-v1.0` |
|
44 |
+
| Fine-Tuning Method | QLoRA via `peft` |
|
45 |
+
| Epochs | 3 (with early stopping) |
|
46 |
+
| Adapter Size | ~7MB (LoRA weights only) |
|
47 |
+
| Hardware | Local CPU (low-resource) |
|
48 |
+
| Tokenizer | Inherited from base model |
|
49 |
+
|
50 |
+
---
|
51 |
|
52 |
## π Evaluation
|
53 |
+
|
54 |
+
| Metric | Result |
|
55 |
+
|----------------------------|----------------|
|
56 |
+
| Accuracy on Eval Set | ~92% |
|
57 |
+
| Manual Review | High relevance |
|
58 |
+
| Hallucination Rate | Very low |
|
59 |
+
| Inference Time (CPU) | < 1s / query |
|
60 |
+
|
61 |
+
All results are stored in `eval_results.json`.
|
62 |
+
|
63 |
+
---
|
64 |
|
65 |
## π§ Files Included
|
|
|
|
|
|
|
|
|
66 |
|
67 |
+
- `adapter_model.safetensors` β fine-tuned LoRA weights
|
68 |
+
- `adapter_config.json` β LoRA hyperparameters
|
69 |
+
- `training.ipynb` β complete training notebook
|
70 |
+
- `agent.py` β CLI interface to test the model
|
71 |
+
- `cli_questions.json` β training dataset
|
72 |
+
- `eval_results.json` β eval results
|
73 |
+
- `requirements.txt` β dependencies
|
74 |
+
|
75 |
+
---
|
76 |
+
|
77 |
+
## π¦ Inference Example
|
78 |
|
79 |
```python
|
80 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
81 |
+
from peft import PeftModel
|
82 |
|
83 |
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
|
84 |
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
|