Regularizing the Deep Image Prior with a Learned Denoiser for Linear Inverse Problems
 

Rita Fermanian, Mikael Le Pendu, Christine Guillemot,
"Regularizing the Deep Image Prior with a Learned Denoiser for Linear Inverse Problems", IEEE MulMedia Signal Processing (MMSP) workshop, accepted, 6-8 Oct. 2021.[preprint]
contact: R. Fermanian, M. Le Pendu, C. Guillemot

Abstract

We propose an optimization method coupling a learned denoiser with the untrained generative model, called deep image prior (DIP) in the framework of the Alternating Direction Method of Multipliers (ADMM) method. We also study different regularizers of DIP optimization, for inverse problems in imaging, focusing in particular on denoising and super-resolution. The goal is to make the best of the untrained DIP and of a generic regularizer learned in a supervised manner from a large collection of images. When placed in the ADMM framework, the denoiser is used as a proximal operator and can be learned independently of the considered inverse problem. We show the benefits of the proposed method, in comparison with other regularized DIP methods, for two linear inverse problems, i.e., denoising and super-resolution.

Results

The algorithm has been tested for two inverse problems: denoising and super-resolution.

Denoising for noise of standard deviation 20



Noisy DIP DIPTV ADMM-DIPTV Denoiser DIP-Denoiser-ADMM
PSNR=27.93 dB PSNR=27.81 dB PSNR=27.96 dB PSNR=31.55 dB PSNR=31.35 dB
PSNR=30.07 dB PSNR=30.52 dB PSNR=30.62 dB PSNR=33.23 dB PSNR=32.96 dB
PSNR=30.01 dB PSNR=31.4 dB PSNR=31.04 dB PSNR=33.7 dB PSNR=33.67 dB
PSNR=29.03 dB PSNR=29.55 dB PSNR=29.3 dB PSNR=32.18 dB PSNR=31.96 dB
PSNR=29.98 dB PSNR=29.63 dB PSNR=29.5 dB PSNR=31.21 dB PSNR=31.1 dB
PSNR=31.29 dB PSNR=32.53 dB PSNR=32.07 dB PSNR=34.02 dB PSNR=33.93 dB

Denoising for noise of standard deviation 30



Noisy DIP DIPTV ADMM-DIPTV Denoiser DIP-Denoiser-ADMM
PSNR=27.93 dB PSNR=27.81 dB PSNR=27.96 dB PSNR=31.55 dB PSNR=31.35 dB
PSNR=30.07 dB PSNR=30.52 dB PSNR=30.62 dB PSNR=33.23 dB PSNR=32.96 dB
PSNR=30.01 dB PSNR=31.4 dB PSNR=31.04 dB PSNR=33.7 dB PSNR=33.67 dB
PSNR=29.03 dB PSNR=29.55 dB PSNR=29.3 dB PSNR=32.18 dB PSNR=31.96 dB
PSNR=29.98 dB PSNR=29.63 dB PSNR=29.5 dB PSNR=31.21 dB PSNR=31.1 dB
PSNR=31.29 dB PSNR=32.53 dB PSNR=32.07 dB PSNR=34.02 dB PSNR=33.93 dB

Super-resolution of factor 4



Downsampled Bicubic DIP DIPTV ADMM-DIPTV DeepRED ADMM-Denoiser DIP-Denoiser-ADMM
PSNR=19.42 dB PSNR=19.72 dB PSNR=19.85 dB PSNR=19.62 dB PSNR=19.88 dB PSNR=20.06 dB PSNR=20.14 dB
PSNR=20.95 dB PSNR=24.23 dB PSNR=23.66 dB PSNR=22.47 dB PSNR=23.79 dB PSNR=24.34 dB PSNR=24.42 dB
PSNR=22.58 dB PSNR=23.23 dB PSNR=23.27 dB PSNR=22.99 dB PSNR=23.34 dB PSNR=23.34 dB PSNR=23.51 dB
PSNR=23.09 dB PSNR=23.05 dB PSNR=23.37 dB PSNR=23.17 dB PSNR=23.36 dB PSNR=23.46 dB PSNR=23.45 dB
PSNR=23.00 dB PSNR=24.13 dB PSNR=24.31 dB PSNR=23.84 dB PSNR=24.69 dB PSNR=24.71 dB PSNR=24.9 dB
PSNR=26.37 dB PSNR=28.44 dB PSNR=27.61 dB PSNR=27.01 dB PSNR=27.81 dB PSNR=28.75 dB PSNR=28.97 dB

Super-resolution of factor 8



Downsampled Bicubic DIP DIPTV ADMM-DIPTV DeepRED ADMM-Denoiser DIP-Denoiser-ADMM
PSNR=16.25 dB PSNR=17.16 dB PSNR=17.32 dB PSNR=17.11 dB PSNR=17.22 dB PSNR=16.83 dB PSNR=17.21 dB
PSNR=15.38 dB PSNR=17.04 dB PSNR=16.62 dB PSNR=16.63 dB PSNR=16.80 dB PSNR=17.11 dB PSNR=17.4 dB
PSNR=18.94 dB PSNR=20.2 dB PSNR=20.1 dB PSNR=19.95 dB PSNR=20.15 dB PSNR=19.88 dB PSNR=20.19 dB
PSNR=19.87 dB PSNR=20.6 dB PSNR=20.65 dB PSNR=20.45 dB PSNR=20.65 dB PSNR=20.47 dB PSNR=20.57 dB
PSNR=16.84 dB PSNR=18.01 dB PSNR=18.15 dB PSNR=17.75 dB PSNR=18.05 dB PSNR=17.37 dB PSNR=17.84 dB
PSNR=20.75 dB PSNR=22.45 dB PSNR=22.1 dB PSNR=22.01 dB PSNR=22.43 dB PSNR=23.46 dB PSNR=23.58 dB