Blind deblurring from synthetic data

MVA project 2024 on image restoration

Jamy Lafenetre
Balthazar Neveu

In the paper Synthetic images as a regularity prior for image restoration neural networks by Raphaël Achddou, Yann Gousseau, Saïd Ladjal, it was shown that a deep neural network can be trained for denoising solely from synthetic dead leave images and does perform relatively well on natural images.

We propose to explore several tracks:

Use a NAFNET architecture
Extend deadleaves with extra primitives (lines, diamond shapes, colored gradients)
Try to see if the generalization property to natural images observed in denoising holds for deblurring.

We first validated that NAFNet trained on deadleaves performed well on the blind denoising task. Below you can see that it also performs correctly on natural images, although the performances are not as good as a network purely trained on natural images.

Qualitative results at SNR in = 20dB	Quantitative results

Finally, when applying the deadleaves training to the blind deblurring problem, one of the advantage we have notticed is that the network always tries to deblur even when the level of blur is high. On the contrary, when trained on natural images, the NAFNEt does not work so well when the blur level is too big.

Blind deblurring results

Deblurring result for different amount of blur, using Nafnet trained on Div2K or deadleaves. From left to right column: ”small”, ”mild” and ”big” blur kernels to degrade the input. Top row: input image. Middle row: output of NafNet trained on deadleaves. Bottom row: output of NafNet trained on Div2K.

Conclusion :

Using extra primitives to pure deadleaves seems like a good idea but did not bring as much as we'd expected. A rework by adding anisotropy and extra geometric shapes could lead to significantly better results.
Training on deadleaves images brings a lot of stability in trainings (as the image distribution is basically always the same, providing a fair amount of learning signal) and can be seen as a good pretext task before to the real dataset.