File size: 2,641 Bytes
923e020 8d309d8 923e020 664e70f 923e020 8d309d8 923e020 ff63da4 923e020 8d309d8 923e020 ff63da4 923e020 1a50fd3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
---
license: cc-by-nc-4.0
library_name: diffusers
---
It's simple upscaler using AsymmetricAutoencoderKL. I was playing around with code used for training in the middle of it a lot so it's nothing scientific. I was just pleased with results from something that easy to train.
For optimizers, training was done with AdEMAMix optimizer, dataset of ~4k images mostly including photos, digital art and small amount of PBR textures. I did some finetuning with same dataset, but Adopt optimizer with OrthoGrad from <a href="https://arxiv.org/abs/2501.04697" target="_blank"><i>Grokking at the Edge of Numerical Stability</i></a> (arXiv: 2501.04697). Model was trained at 96px x 96px resolution (so 192px x 192ox output).
For loss, I was using most of the time simple HSL loss (1 - cosine of difference between target and pred H and L1 loss for S and L channels), LPIPS+ and DISTS.
Model have issues with handling jpeg artifacts because I couldn't train it on random compression levels due to lack of support of ROCm by torchvision.transforms.v2.JPEG. In this case it's better to scale down image a bit before upscaling.
This is some proof of concept model. It can't be used commercially as is, but there is a chance that I'll train new version on some CC0 dataset with license permiting commercial usage and with better jpeg artifacts handling in future.
You can run model using code below
```
import torch
from torchvision import transforms, utils
import diffusers
from diffusers import AsymmetricAutoencoderKL
from diffusers.utils import load_image
def crop_image_to_nearest_divisible_by_8(img):
# Check if the image height and width are divisible by 8
if img.shape[1] % 8 == 0 and img.shape[2] % 8 == 0:
return img
else:
# Calculate the closest lower resolution divisible by 8
new_height = img.shape[1] - (img.shape[1] % 8)
new_width = img.shape[2] - (img.shape[2] % 8)
# Use CenterCrop to crop the image
transform = transforms.CenterCrop((new_height, new_width), interpolation=transforms.InterpolationMode.BILINEAR)
img = transform(img).to(torch.float32).clamp(-1, 1)
return img
to_tensor = transforms.ToTensor()
vae = AsymmetricAutoencoderKL.from_pretrained("Heasterian/AsymmetricAutoencoderKLUpscaler", weight_dtype=torch.float32)
vae.requires_grad_(False)
image = load_image(r"/home/heasterian/test/a/F8VlGmCWEAAUVpc (copy).jpeg")
image = crop_image_to_nearest_divisible_by_8(to_tensor(image)).unsqueeze(0)
upscaled_image = vae(image).sample
# Save the reconstructed image
utils.save_image(upscaled_image, "test.png")
```
|