Update README.md

README.md

@@ -17,4 +17,211 @@ tags:
 - llama
 - llama-3
 license: llama3.2
+---
+
+# Evolution Learning Network (ELN) with QLoRA and Genetic Algorithms For LLM
+
+## Overview
+
+This project implements an **Evolution Learning Network (ELN)** to fine-tune transformer-based models like LLaMA using a combination of **Quantized Low-Rank Adaptation (QLoRA)** and **Genetic Algorithms (GA)**. The primary objective is to evolve a population of models across multiple generations to optimize for performance (fitness) and specialization, while maintaining diversity.
+
+### Key Features
+- Efficient model fine-tuning using **QLoRA** with 4-bit quantization
+- Evolutionary strategies with tournament selection and blended crossover
+- Adaptive mutation rates based on generation progress
+- Comprehensive experiment tracking with **WandB**
+- Diversity maintenance through LoRA weight fingerprinting
+
+## Model Details
+
+### Base Model
+- **Name**: [meta-llama/Llama-3.2-1B](https://huggingface.co/meta-llama/Llama-3.2-1B)
+- **Architecture**: Transformer-based causal language model
+
+### Quantization Configuration
+- **Type**: 4-bit quantization using `bitsandbytes`
+- **Parameters**:
+  - Compute Type: `torch.float16`
+  - Quantization Type: `"nf4"` (Nonlinear)
+  - Double Quantization: Enabled
+  - Nested Quantization: Enabled
+
+### LoRA Configuration
+- **Rank (r)**: 8
+- **Alpha**: 16
+- **Target Modules**: `q_proj`, `v_proj`
+- **Dropout**: 0.05
+- **Task Type**: `CAUSAL_LM`
+
+### Training Configuration
+- **Optimizer**: `paged_adamw_8bit`
+- **Precision**: Mixed precision (`fp16`)
+- **Batch Size Range**: 2-16 (genome-controlled)
+- **Learning Rate Range**: 1e-6 to 1e-2 (genome-controlled)
+- **Epochs Range**: 1-4 (genome-controlled)
+
+## Dataset
+
+### Source
+- **Name**: WikiText-2 Raw
+- **Configuration**: `wikitext-2-raw-v1`
+- **Processing**:
+  - Max Length: 128 tokens
+  - Padding: Fixed to max length
+  - Splits: train, validation (general), test (specific)
+
+## Evolution Process
+
+### Population Management
+1. **Initialization**:
+   - Population Size: 6 models
+   - Initial random mutations (20% rate)
+   - Randomized hyperparameter genomes
+
+2. **Selection & Evolution**:
+   - Tournament selection (k=3)
+   - Blended crossover of LoRA weights
+   - Adaptive mutation rates (decreases with generations)
+   - Hyperparameter mutation with controlled ranges
+
+## Experimental Results
+
+### Evolution Progress
+
+The evolutionary learning process was run for 8 generations with a population size of 6 models. The experiment tracked several key metrics across generations:
+
+Evolution Metrics
+<div style="display: flex;">
+  <img src="https://huggingface.co/diabolic6045/ELN-llama-1B-adapter/resolve/main/images/output.png" alt="Evolution Metrics" style="width: 50%;height: 50%;"/></div>
+
+#### Fitness Progression
+- **Initial Performance**: Best fitness started at ~0.480 (Generation 1)
+- **Convergence**: Gradual decline to ~0.476 by Generation 8
+- **Population Stability**: Average fitness closely tracked best fitness after Generation 2, indicating good convergence
+- **Fitness Range**: Maintained between 0.476-0.480 throughout evolution
+
+#### Specialization Trends
+- **High Baseline**: Started at ~0.9975 specialization
+- **Consistency**: Fluctuated minimally between 0.9975-0.9990
+- **Peak Performance**: Reached ~0.9991 specialization in Generation 6
+- **Population Average**: Maintained above 0.997 throughout evolution
+
+### Comparison with Standard Training
+
+![Training Comparison](https://huggingface.co/diabolic6045/ELN-llama-1B-adapter/resolve/main/images/comparison.webp)
+
+The comparison reveals several key differences between ELN and standard training:
+
+#### Fitness Metrics
+- **ELN**: 0.4762 final fitness with stable progression
+- **Standard**: 0.4779 final fitness with steeper learning curve
+- **Difference**: ~0.3% performance gap, favoring standard training
+
+#### Training Characteristics
+- **Loss Reduction**:
+  - Standard: Sharp initial drop followed by gradual improvement
+  - ELN: More controlled, stable descent
+- **Specialization**:
+  - Standard: More variable specialization scores
+  - ELN: Consistently high specialization maintenance
+
+#### Key Advantages of ELN
+1. More stable learning trajectory
+2. Better maintenance of model diversity
+3. Consistent specialization scores
+4. Reduced risk of catastrophic forgetting
+
+## Hardware & Framework Requirements
+
+### Hardware
+- Multi-GPU support via `DistributedDataParallel`
+- Memory optimization through gradient accumulation
+- Hardware monitoring (CPU/GPU usage)
+
+### Dependencies
+- transformers
+- peft
+- bitsandbytes
+- accelerate
+- wandb
+- torch >= 2.0
+
+## Usage
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM
+
+base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B")
+model = PeftModel.from_pretrained(base_model, "diabolic6045/ELN-llama-1B-adapter")
+```
+
+## Framework Versions
+- PEFT 0.14.0
+
+## Future Work
+- Explore larger population sizes and generations
+- Implement additional mutation strategies
+- Test on diverse datasets and tasks
+- Investigate multi-objective optimization
+
+---
+
+## Citation
+
+If you use this work, please cite:
+
+```bibtex
+@misc{eln2024,
+  title={Evolution Learning Network (ELN): Combining QLoRA and Genetic Algorithms for LLM Optimization},
+  year={2024},
+  howpublished={\url{https://github.com/diabolic6045/ELN-llama-1B-adapter}}
+}
+```
+
+### Related Works
+
+This project builds upon several key papers and techniques:
+
+```bibtex
+@article{dettmers2023qlora,
+  title={QLoRA: Efficient Finetuning of Quantized LLMs},
+  author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
+  journal={arXiv preprint arXiv:2305.14314},
+  year={2023}
+}
+
+@article{touvron2023llama,
+  title={Llama: Open and Efficient Foundation Language Models},
+  author={Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timoth{\'e}e and Rozi{\`e}re, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},
+  journal={arXiv preprint arXiv:2302.13971},
+  year={2023}
+}
+
+@article{such2017deep,
+  title={Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning},
+  author={Such, Felipe Petroski and Madhavan, Vashisht and Conti, Edoardo and Lehman, Joel and Stanley, Kenneth O and Clune, Jeff},
+  journal={arXiv preprint arXiv:1712.06567},
+  year={2017}
+}
+
+@article{real2019regularized,
+  title={Regularized Evolution for Image Classifier Architecture Search},
+  author={Real, Esteban and Aggarwal, Alok and Huang, Yanping and Le, Quoc V},
+  journal={Proceedings of the AAAI Conference on Artificial Intelligence},
+  volume={33},
+  number={01},
+  pages={4780--4789},
+  year={2019}
+}
+```
+
+These citations cover:
+1. QLoRA quantization and fine-tuning technique
+2. The base LLaMA model architecture
+3. Deep neuroevolution fundamentals
+4. Regularized evolution in neural networks
+
+The implementation also draws inspiration from recent advances in evolutionary algorithms and neural architecture search.
+
 ---