File size: 5,649 Bytes
111cffc e120101 291c906 e120101 bee8d68 e120101 bee8d68 3d6518e 291c906 111cffc bee8d68 111cffc bee8d68 111cffc bee8d68 111cffc e120101 111cffc bee8d68 d3ffb35 bee8d68 111cffc bee8d68 111cffc bee8d68 111cffc bee8d68 ba2fab8 111cffc e120101 111cffc bee8d68 bb8b10f e120101 204d564 e120101 204d564 ba2fab8 bee8d68 f04d96d 3d6518e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 |
---
license: creativeml-openrail-m
tags:
- keras
- diffusers
- stable-diffusion
- text-to-image
- diffusion-models-class
- keras-dreambooth
- nature
inference: true
widget:
- text: a photo of puggieace dog on the beach, sunset in background
datasets:
- nielsgl/dreambooth-ace
library_name: keras
pipeline_tag: text-to-image
emoji: 🐶
---
# KerasCV Stable Diffusion in Diffusers 🧨🤗
DreamBooth model for the `puggieace` concept trained by nielsgl on the `nielsgl/dreambooth-ace` dataset.
It can be used by modifying the `instance_prompt`: **a photo of puggieace**.
The examples are from 2 different Keras CV models (`StableDiffusion` and `StableDiffusionV2`, corresponding to Stable Diffusion V1.4 and V2.1, respectively) trained on the same dataset (`nielsgl/dreambooth-ace`).
## Description
The Stable Diffusion V2 pipeline contained in the corresponding repository (`nielsgl/dreambooth-keras-pug-ace-sd2.1`) was created using a modified version of [this Space](https://huggingface.co/spaces/sayakpaul/convert-kerascv-sd-diffusers) for StableDiffusionV2 from KerasCV. The purpose is to convert the KerasCV Stable Diffusion weights in a way that is compatible with [Diffusers](https://github.com/huggingface/diffusers). This allows users to fine-tune using KerasCV and use the fine-tuned weights in Diffusers taking advantage of its nifty features (like [schedulers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/schedulers), [fast attention](https://huggingface.co/docs/diffusers/optimization/fp16), etc.).
This model was created as part of the Keras DreamBooth Sprint 🔥. Visit the [organisation page](https://huggingface.co/keras-dreambooth) for instructions on how to take part!
## Examples
### Stable Diffusion V1.4
> Portrait of puggieace dog as a Roman Emperor, city in background
> Photo of puggieace dog wearing sunglasses on the beach, sunset in background, golden hour
> Photo of cute puggieace dog as an astronaut, planet and spaceship in background
### Stable Diffusion V2.1
> Portrait painting of a cute puggieace dog as a samurai
> Photo of cute puggieace dog as an astronaut, space and planet in background
> A photo of a cute puggieace dog getting a haircut in a barbershop
> Portrait photo of puggieace dog in New York
> Portrait of puggieace dog as a Roman Emperor, city in background
## Usage with Stable Diffusion V1.4
```python
from huggingface_hub import from_pretrained_keras
import keras_cv
import matplotlib.pyplot as plt
model = keras_cv.models.StableDiffusion(img_width=512, img_height=512, jit_compile=True)
model._diffusion_model = from_pretrained_keras("nielsgl/dreambooth-pug-ace")
model._text_encoder = from_pretrained_keras("nielsgl/dreambooth-pug-ace-text-encoder")
images = model.text_to_image("a photo of puggieace dog on the beach", batch_size=3)
plt.imshow(image[0])
```
## Usage with Stable Diffusion V2.1
```python
from diffusers import StableDiffusionPipeline
pipeline = StableDiffusionPipeline.from_pretrained('nielsgl/dreambooth-keras-pug-ace-sd2.1')
image = pipeline().images[0]
image
```
### Training hyperparameters
The following hyperparameters were used during training for Stable Diffusion v1.4:
| Hyperparameters | Value |
| :-- | :-- |
| name | RMSprop |
| weight_decay | None |
| clipnorm | None |
| global_clipnorm | None |
| clipvalue | None |
| use_ema | False |
| ema_momentum | 0.99 |
| ema_overwrite_frequency | 100 |
| jit_compile | True |
| is_legacy_optimizer | False |
| learning_rate | 0.0010000000474974513 |
| rho | 0.9 |
| momentum | 0.0 |
| epsilon | 1e-07 |
| centered | False |
| training_precision | float32 | |