multimodalart
HF staff
update the inference widget text and add instance_prompt for programmable uses
979a3c3
license: mit | |
base_model: stabilityai/stable-diffusion-xl-base-1.0 | |
tags: | |
- stable-diffusion | |
- stable-diffusion-diffusers | |
- text-to-image | |
- diffusers | |
- lora | |
inference: true | |
instance_prompt: sonic the hedgehog | |
widget: | |
- text: sonic the hedgehog | |
# sdxl-ugly-sonic-lora | |
![](img/example1.webp) | |
A LoRA for SDXL 1.0 Base which generates [Ugly Sonic](https://knowyourmeme.com/memes/ugly-sonic), using `sonic the hedgehog` as the trigger keywords. | |
## Usage | |
The LoRA can be loaded using `load_lora_weights` like any other LoRA in `diffusers`: | |
```py | |
import torch | |
from diffusers import DiffusionPipeline, AutoencoderKL | |
vae = AutoencoderKL.from_pretrained( | |
"madebyollin/sdxl-vae-fp16-fix", | |
torch_dtype=torch.float16 | |
) | |
base = DiffusionPipeline.from_pretrained( | |
"stabilityai/stable-diffusion-xl-base-1.0", | |
vae=vae, | |
torch_dtype=torch.float16, | |
variant="fp16", | |
use_safetensors=True | |
) | |
base.load_lora_weights("minimaxir/sdxl-ugly-sonic-lora") | |
_ = base.to("cuda") | |
``` | |
During image generation, use `sonic the hedgehog` in the prompt. | |
## Examples | |
For all generations, the negative prompt used is `blurry, low quality`. Prompt weighting is done using the [compel](https://github.com/damian0815/compel) syntax. | |
`sonic the hedgehog relaxing on a couch, renaissance painting, (oil on canvas, aged, worn)++++` (cfg = 13) | |
![](img/example2.webp) | |
`a profile of sonic the hedgehog sitting at a desk deep in thought, (pixel art)++++, award-winning photo for vanity fair` (cfg = 13) | |
![](img/example3.webp) | |
`anatomical diagram of sonic the hedgehog, (highly detailed)++++` (cfg = 13) | |
![](img/example4.webp) | |
`sonic the hedgehog (eating at McDonald's)++, Ukiyo-e, minimalistic vector art` (cfg = 13) | |
![](img/example5.webp) | |
`(hyperrealistic++ death metal album cover)+++ featuring edgy moody realistic human sonic the hedgehog, edgy and moody` (cfg = 13) | |
![](img/example6.webp) | |
`(hyperrealistic tall human)+++ sonic the hedgehog cameoing in an (episode of Friends)+++++, high quality hyperrealistic promotional photo` (cfg = 13) | |
![](img/example7.webp) | |
## Methodology | |
This LoRA was trained on frame-by-frame analysis of the [original 1080p trailer](https://www.youtube.com/watch?v=4mW9FE5ILJs) featuring "Ugly Sonic". Square-crops of Ugly Sonic were extracted and AI-upscaled to 1024x1024 for optimial training. | |
The use of `sonic the hedgehog` as the trigger keywords (and training the LoRA on said keywords) ensures that you won't generate the _other_ hedgehog by accident. Because no one likes him. | |
## Notes | |
- The CGI style of Ugly Sonic may overpower other style prompts. Therefore, you should weight any style prompts much higher, as done in the examples. | |
- Attempting to generate any humans with this prompt may generate half-hedgehog half-human hybrids. This may be a pro or con depending on what you are going for, but you can specify `human` and weight accordingly. | |