#### NOTE New to DeepInverse? Get started with the basics with the [5 minute quickstart tutorial](https://deepinv.org/auto_examples/basics/demo_quickstart.html.md#sphx-glr-auto-examples-basics-demo-quickstart-py).. # DEAL denoising and reconstruction This example shows how to use the Deep Equilibrium Attention Least Squares (DEAL) model in DeepInverse for both denoising and a simple reconstruction settings. The reconstruction is obtained by solving $$ \hat{x} = \arg\min_x \frac{1}{2}\|Ax - y\|^2 + \lambda g_{\theta}(x), $$ where $A$ is the forward operator, $y$ are the measurements, and $g_{\theta}$ is the learned adaptive regularizer. In DEAL, the regularizer is induced by a masked linear operator $$ L_{\theta,c}(u, x) = m_{\theta,c}(u) \odot K_{\theta,c}x, $$ which depends on an auxiliary image $u$, resulting in $$ g_{\theta}(u, x) = \sum_{c=1}^{C} \frac{1}{2} \|m_{\theta,c}(u) \odot K_{\theta,c}x\|_2^2. $$ The fixed-point iterations used by the solver are $$ x^{(k+1)} \approx \arg\min_x \frac{1}{2}\|Ax-y\|^2 + \lambda \nabla_x g_{\theta}(u=x^{(k)}, x)^\top x. $$ Each subproblem is solved approximately with conjugate gradient. DEAL solves inverse problems by minimizing a data-fidelity term together with a learned adaptive regularizer. In the implementation, this regularizer is induced by a masked linear operator, where learned filters are modulated by spatially varying masks predicted by the network. The reconstruction is then refined through iterative least-squares updates, where the regularizer is recomputed from the current iterate. This implementation is adapted from the official [DEAL repository](https://github.com/mehrsapo/DEAL). Here, the model is illustrated first for Gaussian denoising, and then for a simple inpainting reconstruction problem. ```Python # Import packages and load a grayscale example image. import torch from deepinv.loss.metric import PSNR from deepinv.models import DEAL from deepinv.physics import Denoising, GaussianNoise, Inpainting from deepinv.utils import load_example, plot device = "cuda" if torch.cuda.is_available() else "cpu" torch.manual_seed(0) # Load image (grayscale) x = load_example("butterfly.png", img_size=128, device=device, grayscale=True) ``` Noise level in the normalized [0,1] convention used by DeepInverse. ```Python sigma = 0.1 ``` Load pretrained DEAL model ```Python model = DEAL( pretrained="download", sigma_denoiser=sigma, lambda_reg=10.0, auto_scale=False, color=False, device=device, clamp_output=True, inner_iter=10, outer_iter=10, ) model.eval() n_params = sum(p.numel() for p in model.parameters()) print(f"DEAL number of parameters: {n_params:,}") psnr = PSNR() ``` ```none DEAL number of parameters: 468,570 ``` ## Denoising with DEAL We first illustrate Gaussian denoising with DEAL. The model is applied with a denoising operator, and the plotted mask corresponds to the spatially varying regularization weights from the last iteration. ```Python physics_denoise = Denoising(GaussianNoise(sigma=sigma)).to(device) y_denoise = physics_denoise(x) with torch.no_grad(): x_hat_denoise = model(y_denoise, physics_denoise) mask_denoise = model.mask.mean(dim=1, keepdim=True) psnr_noisy = psnr(y_denoise, x).item() psnr_denoise = psnr(x_hat_denoise, x).item() print(f"[Denoising] PSNR noisy: {psnr_noisy:.2f} dB") print(f"[Denoising] PSNR DEAL: {psnr_denoise:.2f} dB") # The mask corresponds to the spatially varying weights learned by DEAL. # It controls how strongly the regularizer acts at each pixel. plot( [x, y_denoise, x_hat_denoise, mask_denoise], titles=[ "Ground truth", "Noisy input", "DEAL denoising", "DEAL mask", ], subtitles=[ "", f"PSNR: {psnr_noisy:.2f} dB", f"PSNR: {psnr_denoise:.2f} dB", "Mean over channels", ], figsize=(11, 3), ) ``` ```none [Denoising] PSNR noisy: 19.99 dB [Denoising] PSNR DEAL: 29.18 dB ``` ## Reconstruction example with inpainting We next illustrate a simple inpainting problem where a random subset of pixels is removed. DEAL combines data fidelity and its learned adaptive regularization to recover the missing content. ```Python # Create random mask (50% missing pixels) mask = (torch.rand(1, 1, 128, 128, device=device) > 0.5).float() physics_inpaint = Inpainting( img_size=(1, 128, 128), mask=mask, noise_model=GaussianNoise(sigma=0.0), ).to(device) y_inpaint = physics_inpaint(x) with torch.no_grad(): x_lin = physics_inpaint.A_adjoint(y_inpaint) x_hat_inpaint = model(y_inpaint, physics_inpaint) mask_inpaint = model.mask.mean(dim=1, keepdim=True) psnr_meas_inpaint = psnr(y_inpaint, x).item() psnr_lin_inpaint = psnr(x_lin, x).item() psnr_deal_inpaint = psnr(x_hat_inpaint, x).item() print(f"[Inpainting] PSNR measurement: {psnr_meas_inpaint:.2f} dB") print(f"[Inpainting] PSNR linear: {psnr_lin_inpaint:.2f} dB") print(f"[Inpainting] PSNR DEAL: {psnr_deal_inpaint:.2f} dB") # Again, the mask shows the learned spatial weighting of the regularizer. plot( [x, y_inpaint, x_lin, x_hat_inpaint, mask_inpaint], titles=[ "Ground truth", "Masked measurement", "Adjoint baseline", "DEAL reconstruction", "DEAL mask", ], subtitles=[ "", f"PSNR: {psnr_meas_inpaint:.2f} dB", f"PSNR: {psnr_lin_inpaint:.2f} dB", f"PSNR: {psnr_deal_inpaint:.2f} dB", "Mean over channels", ], figsize=(13, 3), ) ``` ```none [Inpainting] PSNR measurement: 6.94 dB [Inpainting] PSNR linear: 6.94 dB [Inpainting] PSNR DEAL: 29.52 dB ``` **Total running time of the script:** (0 minutes 0.704 seconds) 🏷 Tags: Denoising, Inpainting [`Download Jupyter notebook: demo_deal.ipynb`](https://deepinv.org/auto_examples/unfolded/_downloads/7f7bb1b03e8c7d8d65c557e551adab94/demo_deal.ipynb) [`Download Python source code: demo_deal.py`](https://deepinv.org/auto_examples/unfolded/_downloads/c6cd4f997f319ef479102d570e009a92/demo_deal.py) [`Download zipped: demo_deal.zip`](https://deepinv.org/auto_examples/unfolded/_downloads/7ab2c3c2b9e48b4238474481a803db8e/demo_deal.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io)