ahmedheakl/asm2asm-deepseek1.3b-risc

CISC-to-RISC

A fine-tuned version of deepseek-ai/deepseek-coder-1.3b-instruct specialized in converting x86 assembly code to RISCv5-64 assembly.

Model Overview

asm2asm-deepseek1.3b-xtokenizer-risc is designed to assist developers in converting x86 assembly instructions to RISCv5-64 assembly. Leveraging the capabilities of the base model, this fine-tuned variant enhances accuracy and efficiency in assembly code transpilation tasks.

Intended Use

This model is intended for:

• Assembly Code Conversion: Assisting developers in translating x86 assembly instructions to RISCv5-64 architecture.
• Educational Purposes: Helping learners understand the differences and translation mechanisms between x86 and RISCv5-64 assembly.
• Code Optimization: Facilitating optimization processes by converting and refining assembly code across architectures.

Limitations

• Dataset Specificity: The model is fine-tuned on a specific dataset, which may limit its performance on assembly instructions outside the training distribution.
• Complex Instructions: May struggle with highly complex or unconventional assembly instructions not well-represented in the training data.
• Error Propagation: Inaccuracies in the generated RISCv5-64 code can lead to functional discrepancies or bugs if not reviewed.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.PAGED_ADAMW with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 2

Usage

All models and datasets are available on Hugging Face. Below is an example of how to use the best model for converting x86 assembly to RISCv5-64.

Inference Code

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from tqdm import tqdm

# Replace 'hf_token' with your Hugging Face token
hf_token = "your_hf_token_here"

model_name = "ahmedheakl/asm2asm-deepseek1.3b-risc"

instruction = """<｜begin▁of▁sentence｜>You are a helpful coding assistant assistant on converting from x86 to RISCv64 assembly.
### Instruction:
Convert this x86 assembly into RISCv64
```asm
{asm_x86}
"```"
### Response:
```asm
{asm_risc}
"""

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    token=hf_token,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

model.config.use_cache = True

tokenizer = AutoTokenizer.from_pretrained(
    model_name,
    trust_remote_code=True,
    token=hf_token,
)

def inference(asm_x86: str) -> str:
    prompt = instruction.format(asm_x86=asm_x86, asm_risc="")
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
    generated_ids = model.generate(
        **inputs,
        use_cache=True,
        num_return_sequences=1,
        max_new_tokens=8000,
        do_sample=False,
        num_beams=8,
        # temperature=0.7,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
    )
    outputs = tokenizer.batch_decode(generated_ids)[0]
    torch.cuda.empty_cache()
    torch.cuda.synchronize()
    return outputs.split("```asm\n")[-1].split(f"```{tokenizer.eos_token}")[0]


x86 = "DWORD PTR -248[rbp] movsx rdx"
converted_risc = inference(x86)
print(converted_risc)

Experiments and Results

Model	Average Edit Distance (↓)	Exact Match (↑)	Test Accuracy (↑)
GPT4o	1296	0%	8.18%
DeepSeekCoder2-16B	1633	0%	7.36%
Yi-Coder-9B	1653	0%	6.33%
Yi-Coder-1.5B	275	16.98%	49.69%
DeepSeekCoder-1.3B	107	45.91%	77.23%
DeepSeekCoder-1.3B-xTokenizer-int4	119	46.54%	72.96%
DeepSeekCoder-1.3B-xTokenizer-int8	96	49.69%	75.47%
DeepSeekCoder-1.3B-xTokenizer	165	50.32%	79.25%

Table: Comparison of models' performance on the x86 to ARM transpilation task, measured by Edit Distance (lower is better), Exact Match (higher is better), and Test Accuracy (higher is better). The top section lists pre-existing models, while the bottom section lists models trained by us. The best results in each metric are highlighted in bold.

Model	Average Edit Distance (↓)	Exact Match (↑)	Test Accuracy (↑)
GPT4o	1293	0%	7.55%
DeepSeekCoder2-16B	1483	0%	6.29%
DeepSeekCoder-1.3B-xTokenizer-int4	112	14.47%	68.55%
DeepSeekCoder-1.3B-xTokenizer-int8	31	69.81%	88.05%
DeepSeekCoder-1.3B-xTokenizer	27	69.81%	88.68%

Table: Comparison of models' performance on the x86 to RISCv64 transpilation task. The top section lists pre-existing models, while the bottom section lists models trained by us.

Framework versions

• Transformers 4.46.0
• Pytorch 2.4.0
• Datasets 3.0.2
• Tokenizers 0.20.1

Please see paper & code for more information:

• https://github.com/ahmedheakl/asm2asm
• https://arxiv.org/abs/2411.16341

Citations

If you use this model in your research, please cite it as follows:

@article{heakl2024cisc,
  title={From CISC to RISC: language-model guided assembly transpilation},
  author={Heakl, Ahmed and Abi, Chaimaa and Hossam, Rania and Mahmoud, Abdulrahman},
  journal={arXiv preprint arXiv:2411.16341},
  year={2024}
}