README.md

metadata

base_model:
  - apple/DiffuCoder-7B-Base
tags:
  - code
  - text-diffusion-model
  - diffusion large language model
license: unknown

DiffuCoder-7B-Instruct

The DiffuCoder-7B-Instruct model builds on the DiffuCoder-7B-Base checkpoint with instruction-tuning to better follow code-related prompts.

• Training recipe: with a newly introduced pad token, we train this model with fixed length conditionally on OpenCoder-SFT data for 5 epochs.

• Benchmarks: Demonstrates stronger instruction-following capabilities than the Base model.

More details and usage examples:

• Paper: DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation

• GitHub: https://github.com/apple/ml-diffucoder

Acknowledgement

To power this HuggingFace model release, we reuse Dream's modeling architecture and generation utils.