#### NOTE New to DeepInverse? Get started with the basics with the [5 minute quickstart tutorial](#sphx-glr-auto-examples-basics-demo-quickstart-py).. # 5 minute quickstart tutorial Follow this example to get started with DeepInverse in under 5 minutes. **Contents** 1. [Install](#install) 2. [Physics](#physics) 3. [Models](#models) 4. [Datasets](#datasets) 5. [What’s next](#what-s-next) ## 1. Install First, install and import the latest stable release of `deepinv`: ### pip ```bash pip install deepinv ``` ### uv ```bash uv pip install deepinv ``` ### pixi ```bash pixi add --pypi deepinv ``` ### conda ```bash conda install -c conda-forge deepinv ``` We then get the device (CPU in the case of this example). ```Python import deepinv as dinv import torch device = dinv.utils.get_device() ``` ```none Selected GPU 0 with 30936.0 MiB free memory ``` ## 2. Physics In DeepInverse, `x` are images: ```Python x = dinv.utils.load_example("butterfly.png", device=device) ``` Images are tensors of shape `B, C, ...` where `B` is batch size, `C` are channels and `...` are spatial dimensions: ```Python print(x.shape) ``` ```none torch.Size([1, 3, 256, 256]) ``` [Imaging forward operators](https://deepinv.org/user_guide/physics/intro.html.md#physics-intro) are called `physics` and simulate measurements `y` from `x`. ```Python physics = dinv.physics.Inpainting(x.shape[1:], mask=0.3, device=device) y = physics(x) ``` DeepInverse implements [many different types of physics](https://deepinv.org/user_guide/physics/physics.html.md#physics) across various imaging modalities. Physics also possess noise models such as Gaussian or Poisson noise. #### HINT Do you get a matplotlib LaTeX error when plotting? Disable LaTeX using `dinv.utils.disable_tex()` ```Python physics.noise_model = dinv.physics.GaussianNoise(sigma=0.1) y = physics(x) dinv.utils.plot({"GT": x, "Noisy Inpainting \nMeasurement": y}) ``` Many physics also take [physics parameters](https://deepinv.org/user_guide/physics/intro.html.md#parameter-dependent-operators) such as `mask`, `filter`, `sigma` etc.: ```Python # Blur with Gaussian filter parameter filter = dinv.physics.functional.gaussian_blur(sigma=(5, 5)) physics = dinv.physics.BlurFFT(x.shape[1:], filter=filter, device=device) # Simulate measurements y = physics(x) ``` You can easily use your own params by passing these into the `physics`, or you can use a `generator` to [generate random params](https://deepinv.org/user_guide/physics/intro.html.md#physics-generators): ```Python # Blur kernel random generator physics_generator = dinv.physics.generator.MotionBlurGenerator( psf_size=(31, 31), num_channels=3, device=device ) # Generate a dict of random params {"filter": ...} params = physics_generator.step() # Update physics during forward call y2 = physics(x, **params) dinv.utils.plot( { "GT": x, "Blurred...": y, "... with Gaussian kernel": filter, "Blurred ...": y2, "...with motion kernel": params["filter"], } ) ``` ```none /local/jtachell/deepinv/deepinv/examples/basics/demo_quickstart.py:123: DeprecationWarning: Argument 'num_channels' is deprecated and will be removed in a future version. physics_generator = dinv.physics.generator.MotionBlurGenerator( ``` Physics are powerful objects and [have many methods](https://deepinv.org/user_guide/physics/intro.html.md#physics-intro), for example a pseudo-inverse: ```Python # You can also update params like so physics.update(filter=filter.to(device)) x_pinv = physics.A_dagger(y) ``` As it is well-known in the field of inverse problems, the pseudo-inverse can give good results if the problem is noiseless, but it completely fails in the presence of noise - this is why we need reconstructors! ```Python physics.noise_model = dinv.physics.GaussianNoise(sigma=0.1) y = physics(x) x_pinv_noise = physics.A_dagger(y) dinv.utils.plot( {"Pseudoinv \nw/o noise": x_pinv, "Pseudoinv \nwith noise": x_pinv_noise} ) ``` #### TIP Want to use DeepInverse with your own physics operator? Check out [Bring your own physics](https://deepinv.org/auto_examples/basics/demo_custom_physics.html.md#sphx-glr-auto-examples-basics-demo-custom-physics-py) for a tutorial! ## 3. Models In DeepInverse, a `model` is a reconstruction algorithm that **reconstructs** images from `y` and knowledge of `physics`. #### TIP Many models, such as [`Reconstruct Anything Model`](https://deepinv.org/api/stubs/deepinv.models.RAM.html.md#deepinv.models.RAM), are [pretrained reconstructors](https://deepinv.org/user_guide/reconstruction/pretrained-models.html.md#pretrained-models) and can be used out of the box. See [Use a pretrained model](https://deepinv.org/auto_examples/basics/demo_pretrained_model.html.md#sphx-glr-auto-examples-basics-demo-pretrained-model-py) for a full example. ```Python model = dinv.models.RAM(pretrained=True, device=device) x_hat = model(y, physics) ``` Plot the image `x`, the measurement `y` and the reconstructed image `x_hat` and compute [metrics](https://deepinv.org/user_guide/training/metric.html.md#metric): ```Python metric = dinv.metric.PSNR() psnr_y = metric(y, x).item() psnr_x_hat = metric(x_hat, x).item() dinv.utils.plot( { "Ground Truth": x, "Measurement": y, "Reconstruction": x_hat, }, subtitles=["PSNR:", f"{psnr_y:.2f} dB", f"{psnr_x_hat:.2f} dB"], ) ``` Some models are only [denoisers](https://deepinv.org/user_guide/reconstruction/denoisers.html.md#denoisers) that **denoise** images from `y` and `sigma`, which can be used to build many [model-based reconstruction algorithms](https://deepinv.org/user_guide/reconstruction/iterative.html.md#iterative). ```Python denoiser = dinv.models.DRUNet(device=device) x_denoised = denoiser(y, sigma=0.1) model = dinv.optim.DPIR(sigma=0.1, denoiser=denoiser, device=device) x_hat = model(y, physics) dinv.utils.plot( { "Ground Truth": x, "Measurement": y, "Denoised": x_denoised, "Reconstruction": x_hat, }, subtitles=[ "PSNR:", f"{dinv.metric.PSNR()(y, x).item():.2f} dB", f"{dinv.metric.PSNR()(x_denoised, x).item():.2f} dB", f"{dinv.metric.PSNR()(x_hat, x).item():.2f} dB", ], ) ``` DeepInverse covers [many frameworks of reconstruction algorithms](https://deepinv.org/user_guide/reconstruction/introduction.html.md#reconstructors) including [deep model architectures](https://deepinv.org/user_guide/reconstruction/deep-reconstructors.html.md#deep-reconstructors), [iterative algorithms](https://deepinv.org/user_guide/reconstruction/iterative.html.md#iterative), [sampling algorithms](https://deepinv.org/user_guide/reconstruction/sampling.html.md#sampling) (e.g. diffusion models), and [unfolded models](https://deepinv.org/user_guide/reconstruction/unfolded.html.md#unfolded). ```Python # Reconstruct Anything Model foundation model model = dinv.models.RAM(pretrained=True, device=device) ``` #### TIP Want to use DeepInverse with your own network? Just inherit from the reconstructor base class [`deepinv.models.Reconstructor`](https://deepinv.org/api/stubs/deepinv.models.Reconstructor.html.md#deepinv.models.Reconstructor)! ## 4. Datasets You can use DeepInverse with [dataset](https://deepinv.org/user_guide/training/datasets.html.md#datasets), for testing or training. First, define a ground-truth dataset. We implement wrappers for [many popular imaging datasets](https://deepinv.org/user_guide/training/datasets.html.md#datasets) across domains including natural images, medical imaging, satellite imaging, etc. #### TIP It’s easy to use your own dataset with DeepInverse. See [Bring your own dataset](https://deepinv.org/auto_examples/basics/demo_custom_dataset.html.md#sphx-glr-auto-examples-basics-demo-custom-dataset-py) for a tutorial. ```Python dataset = dinv.datasets.SimpleFastMRISliceDataset( "data", anatomy="brain", download=True ) ``` ```none 0%| | 0/820534 [00:00 [Datasets](https://deepinv.org/user_guide/training/datasets.html.md#datasets) return either `x`, tuples `x, y` or `x, y, params` of images, measurements, and optional physics parameters. Given a ground-truth dataset, you can simulate a dataset with random physics: ```Python physics = dinv.physics.MRI(device=device) physics_generator = dinv.physics.generator.RandomMaskGenerator( (320, 320), device=device ) path = dinv.datasets.generate_dataset( dataset, physics, save_dir="data", physics_generator=physics_generator, device=device, ) dataset = dinv.datasets.HDF5Dataset( path, load_physics_generator_params=True, use_dict_output=True ) ``` ```none Dataset has been saved at data/dinv_dataset0.h5 /local/jtachell/deepinv/deepinv/deepinv/datasets/datagenerator.py:181: UserWarning: Neither 'split' nor 'train' parameters are provided. Defaulting to loading the 'train' split. This behaviour is likely to change in future versions of the library. warn( ``` You can use this dataset to [test or train](https://deepinv.org/user_guide/training/trainer.html.md#trainer) a model: ```Python import torch dinv.test( model, torch.utils.data.DataLoader(dataset), physics, plot_images=True, device=device, ) ``` ```none /local/jtachell/deepinv/deepinv/deepinv/training/trainer.py:593: UserWarning: Update progress bar frequency of 1 may slow down training on GPU. Consider setting freq_update_progress_bar > 1. warnings.warn( 0%| | 0/2 [00:00 #### TIP Want to use DeepInverse with your own dataset? Check out [Bring your own dataset](https://deepinv.org/auto_examples/basics/demo_custom_dataset.html.md#sphx-glr-auto-examples-basics-demo-custom-dataset-py) for a tutorial! 🎉 Well done, you now know how to use DeepInverse! ## What’s next? - Try more basic examples, including [how to inference a pretrained model](https://deepinv.org/auto_examples/basics/demo_pretrained_model.html.md#sphx-glr-auto-examples-basics-demo-pretrained-model-py), [how to use your own dataset](https://deepinv.org/auto_examples/basics/demo_custom_dataset.html.md#sphx-glr-auto-examples-basics-demo-custom-dataset-py), or [how to use your custom physics operator](https://deepinv.org/auto_examples/basics/demo_custom_physics.html.md#sphx-glr-auto-examples-basics-demo-custom-physics-py). - Checkout a YouTube tutorial on our [YouTube Channel](https://www.youtube.com/@deepinv). - Dive deeper into our full library of examples. - Read the [User Guide](https://deepinv.org/user_guide.html.md#user-guide) for further details on the concepts introduced here. - Want help? [Open an issue](https://github.com/deepinv/deepinv/issues) ask a message on our [Discord](https://discord.gg/qBqY5jKw3p) or get in touch with our [MAINTAINERS](https://github.com/deepinv/deepinv/blob/main/MAINTAINERS.md). **Total running time of the script:** (0 minutes 7.778 seconds) 🏷 Tags: Denoising [`Download Jupyter notebook: demo_quickstart.ipynb`](https://deepinv.org/auto_examples/basics/_downloads/cfbc806ef9032ef2544a9daf472b8836/demo_quickstart.ipynb) [`Download Python source code: demo_quickstart.py`](https://deepinv.org/auto_examples/basics/_downloads/88007f4630a1fad1c99189a1c1ad08c7/demo_quickstart.py) [`Download zipped: demo_quickstart.zip`](https://deepinv.org/auto_examples/basics/_downloads/5a213bd2a40de421ff2beb1a14f9a09d/demo_quickstart.zip) [Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io)