| license: apache-2.0 | |
| base_model: meta-llama/Llama-2-7b-hf | |
| tags: | |
| - fine-tuned | |
| - gt52 | |
| - chatbot | |
| - custom-dataset | |
| language: | |
| - en | |
| pipeline_tag: text-generation | |
| # gpt2-coder | |
| ## Model Description | |
| This is a fine-tuned version of GPT 2 (124.2M parameters) , trained on codeparrot. | |
| ## Training Details | |
| - **Training Data:** [codeparrot] | |
| - **Training Method:** Fine-tuning | |
| - **Training Duration:** [8 hours/days] | |
| - **Hardware:** [V100] | |
| ## Usage | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| # Load model and tokenizer | |
| model = AutoModelForCausalLM.from_pretrained("smoothich/gpt2-coder") | |
| tokenizer = AutoTokenizer.from_pretrained("smoothich/gpt2-coder") | |
| # Generate text | |
| inputs = tokenizer("Hello, how are you?", return_tensors="pt") | |
| outputs = model.generate(**inputs, max_length=100) | |
| response = tokenizer.decode(outputs[0], skip_special_tokens=True) | |
| print(response) | |
| ``` | |
| ## Training Parameters | |
| - Learning Rate: 5e-4 | |
| - Batch Size: 16 | |
| - Gradient Accumulation: 16 | |
| - Epochs: 1 | |
| - Precision: BF16 | |
| ## Evaluation | |
| [Include evaluation metrics if available] | |
| ## License | |
| This model is released under the Apache 2.0 license. | |