🧠 Code Generation Model – Fine-Tuned `Salesforce/codegen-350M-multi`

This repository contains a fine-tuned version of the Salesforce/codegen-350M-multi model. It generates code snippets based on natural language or function signature prompts.

📦 Base Model

Model: Salesforce/codegen-350M-multi
Architecture: Causal LM (Decoder-only Transformer)
Parameters: ~350M
Supports: Python, JavaScript, Java, and more
Quantized: ✅ FP16 using bitsandbytes (optional)

📚 Dataset

Dataset: code_x_glue_cc_code_to_text

Source: Hugging Face Datasets
Description: Dataset of code snippets (in Python) and corresponding natural language docstrings.

from datasets import load_dataset

dataset = load_dataset("code_x_glue_cc_code_to_text", "python")

📊 Evaluation (Scoring)

Metric: BLEU or CodeBLEU (you can also use exact match, ROUGE, etc.)


from datasets import load_metric

bleu = load_metric("bleu")
bleu_score = bleu.compute(predictions=["generated_code"], references=["reference_code"])
print("BLEU Score:", bleu_score)

📁 Folder Structure

finetuned_codegen_350M/ ├── config.json ├── pytorch_model.bin ├── tokenizer_config.json ├── tokenizer.json ├── special_tokens_map.json ├── vocab.json ├── merges.txt ├── training_args.bin └── README.md

💬 Inference Example


from transformers import pipeline

pipe = pipeline("text-generation", model="./finetuned_codegen_350M", device=0)

prompt = "def is_prime(n):"
result = pipe(prompt, max_length=100, do_sample=True)
print(result[0]["generated_text"])