Add library name and pipeline tag

README.md

@@ -1,3 +1,8 @@
+---
+library_name: diffusers
+pipeline_tag: unconditional-image-generation
+---
+
 # Marrying Autoregressive Transformer and Diffusion with Multi-Reference Autoregression <br><sub>Official PyTorch Implementation</sub>
 
 [![arXiv](https://img.shields.io/badge/arXiv%20paper-2506.09482-b31b1b.svg)](https://arxiv.org/pdf/2506.09482)&nbsp;
@@ -22,10 +27,10 @@ This is a PyTorch/GPU implementation of the paper [Marrying Autoregressive Trans
 
 This repo contains:
 
-* 🪐 A simple PyTorch implementation of [TransDiff Model](models/transdiff.py) and [TransDiff Model with MRAR](models/transdiff_mrar.py)
-* ⚡️ Pre-trained class-conditional TransDiff models trained on ImageNet 256x256 and 512x512
-* 💥 A self-contained [notebook](demo.ipynb) for running various pre-trained TransDiff models
-* 🛸 An TransDiff [training and evaluation script](main.py) using PyTorch DDP
+* \ud83e\ude90 A simple PyTorch implementation of [TransDiff Model](models/transdiff.py) and [TransDiff Model with MRAR](models/transdiff_mrar.py)
+* \u26a1\ufe0f Pre-trained class-conditional TransDiff models trained on ImageNet 256x256 and 512x512
+* \ud83d\udca5 A self-contained [notebook](demo.ipynb) for running various pre-trained TransDiff models
+* \ud83d\udef8 An TransDiff [training and evaluation script](main.py) using PyTorch DDP
 
 ## Preparation
 
@@ -71,10 +76,10 @@ Given that our data augmentation consists of simple center cropping and random f
 the VAE latents can be pre-computed and saved to `CACHED_PATH` to save computations during TransDiff training:
 
 ```
-torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 \
-main_cache.py \
---img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 \
---batch_size 128 \
+torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 \\
+main_cache.py \\
+--img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 \\
+--batch_size 128 \\
 --data_path ${IMAGENET_PATH} --cached_path ${CACHED_PATH}
 ```
 
@@ -86,13 +91,13 @@ Run our interactive visualization [demo](demo.ipynb).
 ### Training
 Script for the TransDiff-L 1StepAR setting (Pretrain TransDiff-L with a width of 1024 channels, 800 epochs):
 ```
-torchrun --nproc_per_node=8 --nnodes=8 --node_rank=${NODE_RANK} --master_addr=${MASTER_ADDR} --master_port=${MASTER_PORT} \
-main.py \
---img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \
---model transdiff_large --diffloss_w 1024 \
---diffusion_batch_mul 4 \
---epochs 800 --warmup_epochs 100 --blr 1.0e-4 --batch_size 32 \
---output_dir ${OUTPUT_DIR} --resume ${OUTPUT_DIR} \
+torchrun --nproc_per_node=8 --nnodes=8 --node_rank=${NODE_RANK} --master_addr=${MASTER_ADDR} --master_port=${MASTER_PORT} \\
+main.py \\
+--img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \\
+--model transdiff_large --diffloss_w 1024 \\
+--diffusion_batch_mul 4 \\
+--epochs 800 --warmup_epochs 100 --blr 1.0e-4 --batch_size 32 \\
+--output_dir ${OUTPUT_DIR} --resume ${OUTPUT_DIR} \\
 --data_path ${IMAGENET_PATH}
 ```
 - Training time is ~115h on 64 A100 GPUs with `--batch_size 32`.
@@ -103,25 +108,25 @@ main.py \
 
 Script for the TransDiff-L MRAR setting (Finetune TransDiff-L MRAR with a width of 1024 channels, 40 epochs):
 ```
-torchrun --nproc_per_node=8 --nnodes=8 --node_rank=${NODE_RANK} --master_addr=${MASTER_ADDR} --master_port=${MASTER_PORT} \
-main.py \
---img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \
---model transdiff_large --diffloss_w 1024 --mrar --bf16 \
---diffusion_batch_mul 2 \
---epochs 40 --warmup_epochs 10 --lr 5.0e-5 --batch_size 16 --gradient_accumulation_steps 2 \
---output_dir ${OUTPUT_DIR} --resume ${Transdiff-L_1StepAR_DIR} \
+torchrun --nproc_per_node=8 --nnodes=8 --node_rank=${NODE_RANK} --master_addr=${MASTER_ADDR} --master_port=${MASTER_PORT} \\
+main.py \\
+--img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \\
+--model transdiff_large --diffloss_w 1024 --mrar --bf16 \\
+--diffusion_batch_mul 2 \\
+--epochs 40 --warmup_epochs 10 --lr 5.0e-5 --batch_size 16 --gradient_accumulation_steps 2 \\
+--output_dir ${OUTPUT_DIR} --resume ${Transdiff-L_1StepAR_DIR} \\
 --data_path ${IMAGENET_PATH}
 ```
 Script for the TransDiff-L 512x512 setting (Finetune TransDiff-L 512x512 with a width of 1024 channels, 150 epochs):
 ```
-torchrun --nproc_per_node=8 --nnodes=8 --node_rank=${NODE_RANK} --master_addr=${MASTER_ADDR} --master_port=${MASTER_PORT} \
-main.py \
---img_size 512 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \
---model transdiff_large --diffloss_w 1024 --ema_rate 0.999 --bf16 \
---diffusion_batch_mul 4 \
---epochs 150 --warmup_epochs 10 --lr 1.0e-4 --batch_size 16 --gradient_accumulation_steps 2 \
---only_train_diff \
---output_dir ${OUTPUT_DIR} --resume ${Transdiff-L_1StepAR_DIR} \
+torchrun --nproc_per_node=8 --nnodes=8 --node_rank=${NODE_RANK} --master_addr=${MASTER_ADDR} --master_port=${MASTER_PORT} \\
+main.py \\
+--img_size 512 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \\
+--model transdiff_large --diffloss_w 1024 --ema_rate 0.999 --bf16 \\
+--diffusion_batch_mul 4 \\
+--epochs 150 --warmup_epochs 10 --lr 1.0e-4 --batch_size 16 --gradient_accumulation_steps 2 \\
+--only_train_diff \\
+--output_dir ${OUTPUT_DIR} --resume ${Transdiff-L_1StepAR_DIR} \\
 --data_path ${IMAGENET_PATH}
 ```
 
@@ -129,34 +134,34 @@ main.py \
 
 Evaluate TransDiff-L 1StepAR with classifier-free guidance:
 ```
-torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 \
-main.py \
---img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \
---model transdiff_large --diffloss_w 1024 \
---output_dir ${OUTPUT_DIR} --resume ckpt/transdiff_l/ \
---evaluate --eval_bsz 256 --num_images 50000 \
+torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 \\
+main.py \\
+--img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \\
+--model transdiff_large --diffloss_w 1024 \\
+--output_dir ${OUTPUT_DIR} --resume ckpt/transdiff_l/ \\
+--evaluate --eval_bsz 256 --num_images 50000 \\
 --cfg 1.3 --scale_0 0.89 --scale_1 0.95
 ```
 
 Evaluate TransDiff-L MRAR with classifier-free guidance:
 ```
-torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 \
-main.py \
---img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \
---model transdiff_large --diffloss_w 1024 \
---output_dir ${OUTPUT_DIR} --resume ckpt/transdiff_l_mrar/ \
---evaluate --eval_bsz 256 --num_images 50000 \
+torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 \\
+main.py \\
+--img_size 256 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \\
+--model transdiff_large --diffloss_w 1024 \\
+--output_dir ${OUTPUT_DIR} --resume ckpt/transdiff_l_mrar/ \\
+--evaluate --eval_bsz 256 --num_images 50000 \\
 --cfg 1.3 --scale_0 0.91 --scale_1 0.93
 ```
 
 Evaluate TransDiff-L 512x512 with classifier-free guidance:
 ```
-torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 \
-main.py \
---img_size 512 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \
---model transdiff_large --diffloss_w 1024 \
---output_dir ${OUTPUT_DIR} --resume ckpt/transdiff_l_512/ \
---evaluate --eval_bsz 64 --num_images 50000 \
+torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 \\
+main.py \\
+--img_size 512 --vae_path ckpt/vae/kl16.ckpt --vae_embed_dim 16 --patch_size 1 \\
+--model transdiff_large --diffloss_w 1024 \\
+--output_dir ${OUTPUT_DIR} --resume ckpt/transdiff_l_512/ \\
+--evaluate --eval_bsz 64 --num_images 50000 \\
 --cfg 1.3 --scale_0 0.87 --scale_1 0.87
 ```