#### NOTE
New to DeepInverse? Get started with the basics with the
[5 minute quickstart tutorial](https://deepinv.org/auto_examples/basics/demo_quickstart.html.md#sphx-glr-auto-examples-basics-demo-quickstart-py)..
# Low-intensity STED fluorescence microscopy denoising
This example shows how to denoise low-intensity STED fluorescence microscopy
images of live-cell mitochondria using the pretrained foundation model
[`deepinv.models.RAM`](https://deepinv.org/api/stubs/deepinv.models.RAM.html.md#deepinv.models.RAM). We load real Abberior STED microscopy data
from Osuna-Vargas *et al.*[1](#footcite-osunavargas2025denoising), process it in batches, and
visualize the results both with [`deepinv.utils.plot()`](https://deepinv.org/api/stubs/deepinv.utils.plot.html.md#deepinv.utils.plot) and with the
interactive 3D viewer [`deepinv.utils.plot_napari()`](https://deepinv.org/api/stubs/deepinv.utils.plot_napari.html.md#deepinv.utils.plot_napari).
Acquiring fluorescence microscopy images at low excitation laser power reduces
photobleaching and phototoxicity, which lets biologists image living samples for
longer, but it produces noisier images. Here we recover clean images from
acquisitions taken at 1.5 microwatt excitation, using images taken at 8 microwatt
as the ground truth.
This example requires `tifffile`, `rarfile` and `napari`. Install them with
`pip install tifffile rarfile "napari[all]"`.
```Python
import deepinv as dinv
import torch
device = dinv.utils.get_device()
```
```none
Selected GPU 0 with 15237.25 MiB free memory
```
## Download the dataset
We download the live-cell mitochondria STED dataset from
[Zenodo](https://zenodo.org/records/14215838).
The dataset includes the low-intensity acquisitions (1.5 microwatt) as well as a reference high-intensity (8 microwatt) ground truth, which we use for visualisation.
```Python
data_dir = dinv.utils.get_cache_home() / "datasets" / "sted"
archive_name = "live_cell_mitochondria_u2os_tom20_halotag7_dm_sir"
dinv.datasets.download_archive(
f"https://zenodo.org/records/14215838/files/{archive_name}.rar?download=1",
data_dir / f"{archive_name}.rar",
extract=True,
)
```
```none
File already downloaded: /local/jtachell/.cache/deepinv/datasets/sted/live_cell_mitochondria_u2os_tom20_halotag7_dm_sir.rar. Skipping...
```
## Build the dataset
We use [`deepinv.datasets.ImageFolder`](https://deepinv.org/api/stubs/deepinv.datasets.ImageFolder.html.md#deepinv.datasets.ImageFolder) to pair the ground truth and
low-intensity images. TIFF files are loaded with [`deepinv.utils.load_tiff()`](https://deepinv.org/api/stubs/deepinv.utils.load_tiff.html.md#deepinv.utils.load_tiff).
```Python
dataset = dinv.datasets.ImageFolder(
data_dir / archive_name / "test_and_training_data_1",
x_path="ground_truth_images/*.tif",
y_path="low_intensity_images/*.tif",
loader=lambda f: dinv.utils.load_tiff(f).squeeze(0).float(),
)
```
```none
/local/jtachell/deepinv/deepinv/deepinv/datasets/base.py:495: DeprecationWarning: The tuple format for dataset outputs is deprecated and will be removed in a future version.It is recommended to set `use_dict_output=True` for better readability and flexibility in returned outputs.The default is currently `False` for backward compatibility, but will be switched to `True` in a future version.
super().__init__(use_dict_output=use_dict_output)
```
## Denoise with the RAM foundation model
[`deepinv.models.RAM`](https://deepinv.org/api/stubs/deepinv.models.RAM.html.md#deepinv.models.RAM) is a pretrained foundation model for image
restoration. We process the data in batches. The `gain`
argument sets the Poisson noise level used by the model.
```Python
model = dinv.models.RAM(device=device)
for x, y in torch.utils.data.DataLoader(
dataset, batch_size=1
): # adjust batch size as necessary
x, y = x.to(device), y.to(device)
with torch.no_grad():
rescale = y.max()
x_net = model(y / rescale, gain=0.15) * rescale
break # remove this to process the whole dataset
```
## Visualize the results
We compare the ground truth, the noisy low-intensity measurement, and the
denoised reconstruction.
```Python
dinv.utils.plot(
{
"8 µW excitation": x,
"1.5 µW excitation": y,
"1.5 µW excitation\ndenoised": x_net,
},
figsize=(7, 5),
)
```
## Interactive viewer with napari
Use [`deepinv.utils.plot_napari()`](https://deepinv.org/api/stubs/deepinv.utils.plot_napari.html.md#deepinv.utils.plot_napari) to interactively visualise microscopy images/stacks/volumes
with the [napari](https://napari.org) viewer.
Since it opens an interactive window, it requires a display and is not executed in this gallery.
To reproduce the screenshot below, run:
```python
dinv.utils.plot_napari(x, y, x_net, screenshot=False)
```
Pass `screenshot=True` to take a static screenshot instead:
* **References:**
* Pamela Osuna-Vargas, Maren H. Wehrheim, Lucas Zinz, Johanna Rahm, Ashwin Balakrishnan, Alexandra Kaminer, Mike Heilemann, and Matthias Kaschube. Denoising diffusion models for high-resolution microscopy image restoration. In *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)*. 2025.
**Total running time of the script:** (0 minutes 0.906 seconds)
🏷 Tags: Denoising, Microscopy
[`Download Jupyter notebook: demo_microscopy_denoising.ipynb`](https://deepinv.org/auto_examples/external-libraries/_downloads/1d1b76df953c9d8da6eb675c7824b2b6/demo_microscopy_denoising.ipynb)
[`Download Python source code: demo_microscopy_denoising.py`](https://deepinv.org/auto_examples/external-libraries/_downloads/a79dc6872881b305c7f249e4a8d49b66/demo_microscopy_denoising.py)
[`Download zipped: demo_microscopy_denoising.zip`](https://deepinv.org/auto_examples/external-libraries/_downloads/5520e3fd5af78ef0c893e10837c751d5/demo_microscopy_denoising.zip)
[Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io)