#### NOTE
New to DeepInverse? Get started with the basics with the
[5 minute quickstart tutorial](https://deepinv.org/auto_examples/basics/demo_quickstart.html.md#sphx-glr-auto-examples-basics-demo-quickstart-py)..
# Reconstructing an image using the deep image prior.
This code shows how to reconstruct a noisy and incomplete image using the deep image prior.
This method is based on the paper “Deep Image Prior” Ulyanov *et al.*[1](#footcite-ulyanov2018deep) and reconstructs
an image by minimizing the loss function
$$
\min_{\theta} \|y-Af_{\theta}(z)\|^2
$$
where $z$ is a random input and $f_{\theta}$ is a convolutional decoder network with parameters
$\theta$. The minimization should be stopped early to avoid overfitting. The method uses the Adam
optimizer.
```Python
import deepinv as dinv
from deepinv.utils.plotting import plot
import torch
from deepinv.utils import load_example
```
## Load image from the internet
This example uses an image of Messi.
```Python
device = dinv.utils.get_device()
x = load_example("messi.jpg", img_size=32).to(device)
# Set the global random seed from pytorch to ensure reproducibility of the example.
torch.manual_seed(0)
```
```none
Selected GPU 0 with 28307.25 MiB free memory
```
## Define forward operator and noise model
We use image inpainting as the forward operator and Gaussian noise as the noise model.
```Python
sigma = 0.1 # noise level
physics = dinv.physics.Inpainting(mask=0.5, img_size=x.shape[1:], device=device)
physics.noise_model = dinv.physics.GaussianNoise(sigma=sigma)
```
## Generate the measurement
We apply the forward model to generate the noisy measurement.
```Python
y = physics(x)
```
## Define the deep image prior
This method only works with certain convolutional decoder networks. We recommend using the
network [`deepinv.models.ConvDecoder`](https://deepinv.org/api/stubs/deepinv.models.ConvDecoder.html.md#deepinv.models.ConvDecoder).
#### NOTE
The number of iterations and learning rate have been set manually to obtain good results. However, these
values may not be optimal for all problems. We recommend experimenting with different values.
#### NOTE
Here we run a small number of iterations to reduce the runtime of the example. However, the results could
be improved by running more iterations.
```Python
iterations = 100
lr = 1e-2 # learning rate for the optimizer.
channels = 64 # number of channels per layer in the decoder.
in_size = [2, 2] # size of the input to the decoder.
backbone = dinv.models.ConvDecoder(
img_size=x.shape[1:], in_size=in_size, channels=channels
).to(device)
f = dinv.models.DeepImagePrior(
backbone,
learning_rate=lr,
iterations=iterations,
verbose=True,
img_size=[channels] + in_size,
).to(device)
```
## Run DIP algorithm and plot results
We run the DIP algorithm and plot the results.
The good performance of DIP is somewhat surprising, since the network has many parameters and could potentially
overfit the noisy measurement data. However, the architecture acts as an implicit regularizer, providing good
reconstructions if the optimization is stopped early.
While this phenomenon is not yet well understood, there has been some efforts to explain it. For example, see Tachella *et al.*[2](#footcite-tachella2021neural).
```Python
dip = f(y, physics)
# compute linear inverse
x_lin = physics.A_adjoint(y)
# compute PSNR
psnr_measurement = dinv.metric.PSNR()(x, y).item()
psnr_linear = dinv.metric.PSNR()(x, x_lin).item()
psnr_dip = dinv.metric.PSNR()(x, dip).item()
# plot results
plot(
{
"Ground Truth": x,
"Measurement": y,
"Linear": x_lin,
"DIP": dip,
},
subtitles=[
"PSNR",
f"{psnr_measurement:.2f} dB",
f"{psnr_linear:.2f} dB",
f"{psnr_dip:.2f} dB",
],
)
```
```none
0%| | 0/100 [00:00
* **References:**
* Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In *Proceedings of the IEEE conference on computer vision and pattern recognition*, 9446–9454. 2018.
* Julián Tachella, Junqi Tang, and Mike Davies. The neural tangent link between cnn denoisers and non-local filters. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 8618–8627. 2021.
**Total running time of the script:** (0 minutes 0.290 seconds)
[`Download Jupyter notebook: demo_dip.ipynb`](https://deepinv.org/auto_examples/optimization/_downloads/0120067f2e8f5dec39e8364ef3aa6ea6/demo_dip.ipynb)
[`Download Python source code: demo_dip.py`](https://deepinv.org/auto_examples/optimization/_downloads/1bde39691c908e015c46feac774da9de/demo_dip.py)
[`Download zipped: demo_dip.zip`](https://deepinv.org/auto_examples/optimization/_downloads/b7b3aa527989aa87ea8eb10f7d42fa16/demo_dip.zip)
[Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io)