Eugene Siow
commited on
Commit
•
0a0b275
1
Parent(s):
84f4c1d
Add new training recipe.
Browse files
README.md
CHANGED
@@ -5,6 +5,10 @@ tags:
|
|
5 |
- image-super-resolution
|
6 |
datasets:
|
7 |
- eugenesiow/Div2k
|
|
|
|
|
|
|
|
|
8 |
metrics:
|
9 |
- pnsr
|
10 |
- ssim
|
@@ -42,8 +46,9 @@ preds = model(inputs)
|
|
42 |
ImageLoader.save_image(preds, './scaled_2x.png') # save the output 2x scaled image to `./scaled_2x.png`
|
43 |
ImageLoader.save_compare(inputs, preds, './scaled_2x_compare.png') # save an output comparing the super-image with a bicubic scaling
|
44 |
```
|
|
|
45 |
## Training data
|
46 |
-
The models for 2x, 3x and 4x image super resolution were pretrained on [DIV2K](https://
|
47 |
## Training procedure
|
48 |
### Preprocessing
|
49 |
We follow the pre-processing and training method of [Wang et al.](https://arxiv.org/abs/2104.07566).
|
@@ -51,24 +56,20 @@ Low Resolution (LR) images are created by using bicubic interpolation as the res
|
|
51 |
During training, RGB patches with size of 64×64 from the LR input are used together with their corresponding HR patches.
|
52 |
Data augmentation is applied to the training set in the pre-processing stage where five images are created from the four corners and center of the original image.
|
53 |
|
54 |
-
|
|
|
|
|
|
|
|
|
|
|
55 |
```python
|
56 |
-
from
|
|
|
57 |
|
58 |
-
|
59 |
-
|
60 |
-
|
61 |
-
|
62 |
-
do_augmentation=True
|
63 |
-
)
|
64 |
-
DatasetBuilder.prepare(
|
65 |
-
base_path='./DIV2K/DIV2K_val_HR',
|
66 |
-
output_path='./div2k_4x_val.h5',
|
67 |
-
scale=4,
|
68 |
-
do_augmentation=False
|
69 |
-
)
|
70 |
-
train_dataset = TrainAugmentDataset('./div2k_4x_train.h5', scale=4)
|
71 |
-
val_dataset = EvalDataset('./div2k_4x_val.h5')
|
72 |
```
|
73 |
### Pretraining
|
74 |
The model was trained on GPU. The training code is provided below:
|
@@ -94,18 +95,19 @@ trainer = Trainer(
|
|
94 |
|
95 |
trainer.train()
|
96 |
```
|
|
|
97 |
## Evaluation results
|
98 |
The evaluation metrics include [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio#Quality_estimation_with_PSNR) and [SSIM](https://en.wikipedia.org/wiki/Structural_similarity#Algorithm).
|
99 |
|
100 |
Evaluation datasets include:
|
101 |
-
- Set5 - [Bevilacqua et al. (2012)](
|
102 |
-
- Set14 - [Zeyde et al. (2010)](https://
|
103 |
-
- BSD100 - [Martin et al. (2001)](https://
|
104 |
-
- Urban100 - [Huang et al. (2015)](https://
|
105 |
|
106 |
The results columns below are represented below as `PSNR/SSIM`. They are compared against a Bicubic baseline.
|
107 |
|
108 |
-
|Dataset |Scale |Bicubic |
|
109 |
|--- |--- |--- |--- |
|
110 |
|Set5 |2x |33.64/0.9292 |**37.87/0.9602** |
|
111 |
|Set5 |3x |30.39/0.8678 | |
|
@@ -122,6 +124,10 @@ The results columns below are represented below as `PSNR/SSIM`. They are compare
|
|
122 |
|
123 |
![Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2](images/a2n_2_4_compare.png "Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2")
|
124 |
|
|
|
|
|
|
|
|
|
125 |
## BibTeX entry and citation info
|
126 |
```bibtex
|
127 |
@misc{chen2021attention,
|
|
|
5 |
- image-super-resolution
|
6 |
datasets:
|
7 |
- eugenesiow/Div2k
|
8 |
+
- eugenesiow/Set5
|
9 |
+
- eugenesiow/Set14
|
10 |
+
- eugenesiow/BSD100
|
11 |
+
- eugenesiow/Urban100
|
12 |
metrics:
|
13 |
- pnsr
|
14 |
- ssim
|
|
|
46 |
ImageLoader.save_image(preds, './scaled_2x.png') # save the output 2x scaled image to `./scaled_2x.png`
|
47 |
ImageLoader.save_compare(inputs, preds, './scaled_2x_compare.png') # save an output comparing the super-image with a bicubic scaling
|
48 |
```
|
49 |
+
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Upscale_Images_with_Pretrained_super_image_Models.ipynb "Open in Colab")
|
50 |
## Training data
|
51 |
+
The models for 2x, 3x and 4x image super resolution were pretrained on [DIV2K](https://huggingface.co/datasets/eugenesiow/Div2k), a dataset of 800 high-quality (2K resolution) images for training, augmented to 4000 images and uses a dev set of 100 validation images (images numbered 801 to 900).
|
52 |
## Training procedure
|
53 |
### Preprocessing
|
54 |
We follow the pre-processing and training method of [Wang et al.](https://arxiv.org/abs/2104.07566).
|
|
|
56 |
During training, RGB patches with size of 64×64 from the LR input are used together with their corresponding HR patches.
|
57 |
Data augmentation is applied to the training set in the pre-processing stage where five images are created from the four corners and center of the original image.
|
58 |
|
59 |
+
We need the huggingface [datasets](https://huggingface.co/datasets?filter=task_ids:other-other-image-super-resolution) library to download the data:
|
60 |
+
```bash
|
61 |
+
pip install datasets
|
62 |
+
```
|
63 |
+
The following code gets the data and preprocesses/augments the data.
|
64 |
+
|
65 |
```python
|
66 |
+
from datasets import load_dataset
|
67 |
+
from super_image.data import EvalDataset, TrainDataset, augment_five_crop
|
68 |
|
69 |
+
augmented_dataset = load_dataset('eugenesiow/Div2k', 'bicubic_x4', split='train')\
|
70 |
+
.map(augment_five_crop, batched=True, desc="Augmenting Dataset") # download and augment the data with the five_crop method
|
71 |
+
train_dataset = TrainDataset(augmented_dataset) # prepare the train dataset for loading PyTorch DataLoader
|
72 |
+
eval_dataset = EvalDataset(load_dataset('eugenesiow/Div2k', 'bicubic_x4', split='validation')) # prepare the eval dataset for the PyTorch DataLoader
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
73 |
```
|
74 |
### Pretraining
|
75 |
The model was trained on GPU. The training code is provided below:
|
|
|
95 |
|
96 |
trainer.train()
|
97 |
```
|
98 |
+
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Train_super_image_Models.ipynb "Open in Colab")
|
99 |
## Evaluation results
|
100 |
The evaluation metrics include [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio#Quality_estimation_with_PSNR) and [SSIM](https://en.wikipedia.org/wiki/Structural_similarity#Algorithm).
|
101 |
|
102 |
Evaluation datasets include:
|
103 |
+
- Set5 - [Bevilacqua et al. (2012)](https://huggingface.co/datasets/eugenesiow/Set5)
|
104 |
+
- Set14 - [Zeyde et al. (2010)](https://huggingface.co/datasets/eugenesiow/Set14)
|
105 |
+
- BSD100 - [Martin et al. (2001)](https://huggingface.co/datasets/eugenesiow/BSD100)
|
106 |
+
- Urban100 - [Huang et al. (2015)](https://huggingface.co/datasets/eugenesiow/Urban100)
|
107 |
|
108 |
The results columns below are represented below as `PSNR/SSIM`. They are compared against a Bicubic baseline.
|
109 |
|
110 |
+
|Dataset |Scale |Bicubic |A2N |
|
111 |
|--- |--- |--- |--- |
|
112 |
|Set5 |2x |33.64/0.9292 |**37.87/0.9602** |
|
113 |
|Set5 |3x |30.39/0.8678 | |
|
|
|
124 |
|
125 |
![Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2](images/a2n_2_4_compare.png "Comparing Bicubic upscaling against the models x2 upscaling on Set5 Image 2")
|
126 |
|
127 |
+
You can find a notebook to easily run evaluation on pretrained models below:
|
128 |
+
|
129 |
+
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/eugenesiow/super-image-notebooks/blob/master/notebooks/Evaluate_Pretrained_super_image_Models.ipynb "Open in Colab")
|
130 |
+
|
131 |
## BibTeX entry and citation info
|
132 |
```bibtex
|
133 |
@misc{chen2021attention,
|