Create README.md

README.md

@@ -0,0 +1,42 @@
+---
+license: creativeml-openrail-m
+language:
+- en
+library_name: diffusers
+pipeline_tag: text-to-image
+tags:
+- stable-diffusion
+---
+# 🧩 TokenCompose SD21 Model Card
+
+[TokenCompose_SD21_B](https://mlpc-ucsd.github.io/TokenCompose/) is a [latent text-to-image diffusion model](https://arxiv.org/abs/2112.10752) finetuned from the [**Stable-Diffusion-v2-1**](https://huggingface.co/stabilityai/stable-diffusion-2-1) checkpoint at resolution 768x768 on the [VSR](https://github.com/cambridgeltl/visual-spatial-reasoning) split of [COCO image-caption pairs](https://cocodataset.org/#download) for 32,000 steps with a learning rate of 5e-6. The training objective involves token-level grounding terms in addition to denoising loss for enhanced multi-category instance composition and photorealism. The "_A/B" postfix indicates different finetuning runs of the model using the same above configurations.
+
+# 🧨Example Usage
+
+We strongly recommend using the [🤗Diffuser](https://github.com/huggingface/diffusers) library to run our model.
+
+```python
+import torch
+from diffusers import StableDiffusionPipeline
+
+model_id = "mlpc-lab/TokenCompose_SD21_B"
+device = "cuda"
+
+pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
+pipe = pipe.to(device)
+
+prompt = "A cat and a wine glass"
+image = pipe(prompt).images[0]  
+    
+image.save("cat_and_wine_glass.png")
+```
+
+# ⬆️Improvements over SD21
+
+| Model               | Object Accuracy | MG3 COCO | MG4 COCO | MG5 COCO | MG3 ADE20K | MG4 ADE20K | MG5 ADE20K | FID COCO |
+|---------------------|-----------------|----------|----------|----------|------------|------------|------------|----------|
+| SD21                | 47.82           | 70.14    | 25.57    | 3.27     | 75.13      | 35.07      | 7.16       | 19.59    |
+| TokenCompose (SD21) | 60.10           | 80.48    | 36.69    | 5.71     | 79.51      | 39.59      | 8.13       | 19.15    |
+
+# 📰 Citation
+Coming soon!