README.md

metadata

license: apache-2.0
base_model: meta-llama/Llama-2-7b-hf
tags:
  - fine-tuned
  - gt52
  - chatbot
  - custom-dataset
language:
  - en
pipeline_tag: text-generation

gpt2-coder

Model Description

This is a fine-tuned version of GPT 2 (124.2M parameters) , trained on codeparrot.

Training Details

• Training Data: [codeparrot]
• Training Method: Fine-tuning
• Training Duration: [8 hours/days]
• Hardware: [V100]

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("smoothich/gpt2-coder")
tokenizer = AutoTokenizer.from_pretrained("smoothich/gpt2-coder")

# Generate text
inputs = tokenizer("Hello, how are you?", return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Parameters

• Learning Rate: 5e-4
• Batch Size: 16
• Gradient Accumulation: 16
• Epochs: 1
• Precision: BF16

Evaluation

[Include evaluation metrics if available]

License

This model is released under the Apache 2.0 license.