#### NOTE
New to DeepInverse? Get started with the basics with the
[5 minute quickstart tutorial](https://deepinv.org/auto_examples/basics/demo_quickstart.html.md#sphx-glr-auto-examples-basics-demo-quickstart-py)..
# Scan-specific zero-shot SSDU for MRI
We demonstrate scan-specific self-supervised learning, that is, learning to
reconstruct MRI scans from a single accelerated sample without ground truth.
Here, we demonstrate fine-tuning a pretrained model ([`deepinv.models.RAM`](https://deepinv.org/api/stubs/deepinv.models.RAM.html.md#deepinv.models.RAM)) [1](#footcite-terris2025reconstruct)
with the [`weighted SSDU`](https://deepinv.org/api/stubs/deepinv.loss.mri.WeightedSplittingLoss.html.md#deepinv.loss.mri.WeightedSplittingLoss) loss [2](#footcite-millard2023theoretical)[3](#footcite-yaman2020self).
However, note that any of the [self-supervised losses](https://deepinv.org/user_guide/training/loss.html.md#self-supervised-losses) can be used to do this with varying performance [4](#footcite-wang2025benchmarking).
For example see the [example using Equivariant Imaging](https://deepinv.org/auto_examples/self-supervised-learning/demo_equivariant_imaging.html.md#sphx-glr-auto-examples-self-supervised-learning-demo-equivariant-imaging-py) [5](#footcite-chen2021equivariant).
Note that, if more data is available, better results can be obtained by fine-tuning on more samples!
```Python
import torch
import deepinv as dinv
device = dinv.utils.get_device()
rng = torch.Generator(device=device).manual_seed(0)
rng_cpu = torch.Generator(device="cpu").manual_seed(0)
```
```none
Selected GPU 0 with 22455.25 MiB free memory
```
## Data
First, download a demo single brain MRI volume (FLAIR sequence, SIEMENS Trio Tim 3T scanner)
from the FastMRI brain dataset [6](#footcite-knoll2020advancing), via HuggingFace.
#### IMPORTANT
By using this dataset, you confirm that you have agreed to and signed the [FastMRI data use agreement](https://fastmri.med.nyu.edu/).
```Python
DATA_DIR = dinv.utils.get_cache_home() / "datasets" / "fastMRI" / "multicoil_train"
SLICE_DIR = DATA_DIR / "slices"
DATA_DIR.mkdir(parents=True, exist_ok=True)
SLICE_DIR.mkdir(exist_ok=True)
dinv.utils.download_example("demo_fastmri_brain_multicoil.h5", DATA_DIR)
```
We use the FastMRI slice dataset provided in DeepInverse to load the volume and return all 16 slices.
The data is returned in the format `x, y, params` where `params` is a dictionary containing
the acceleration mask (simulated Gaussian mask with acceleration 6) and the
estimated coil sensitivity map.
#### NOTE
This loading takes a few seconds per slice, as it must estimate the coil sensitivity map
on the fly.
```Python
dataset = dinv.datasets.FastMRISliceDataset(
DATA_DIR,
slice_index="all",
transform=dinv.datasets.MRISliceTransform(
mask_generator=dinv.physics.generator.GaussianMaskGenerator(
img_size=(256, 256), # this is overridden internally by true image size
acceleration=6,
center_fraction=0.08,
device="cpu",
rng=rng_cpu,
),
estimate_coil_maps=True,
),
)
```
```none
/local/jtachell/deepinv/deepinv/deepinv/datasets/fastmri.py:357: DeprecationWarning: The tuple format for dataset outputs is deprecated and will be removed in a future version.It is recommended to set `use_dict_output=True` for better readability and flexibility in returned outputs.The default is currently `False` for backward compatibility, but will be switched to `True` in a future version.
super().__init__(use_dict_output=use_dict_output)
0%| | 0/1 [00:00
When training with data that is slow to be loaded, it is faster to save the pre-loaded slices:
```Python
from tqdm import tqdm
if not any(SLICE_DIR.iterdir()):
for i, (x, y, params) in tqdm(enumerate(dataset)):
torch.save([x, y, params], SLICE_DIR / f"{i}.pt")
```
Then the dataset can be loaded very quickly. We also pre-normalize to bring the data into a more friendly range.
We also load a rough noise level as a param to be passed into the physics.
The ground truth is loaded for evaluation later.
```Python
def loader(f):
x, y, params = torch.load(f, weights_only=True)
return x * 1e5, y * 1e5, params | {"sigma": 1e-5 * 1e5}
dataset = dinv.datasets.ImageFolder(SLICE_DIR, x_path="*.pt", loader=loader)
```
```none
/local/jtachell/deepinv/deepinv/deepinv/datasets/base.py:495: DeprecationWarning: The tuple format for dataset outputs is deprecated and will be removed in a future version.It is recommended to set `use_dict_output=True` for better readability and flexibility in returned outputs.The default is currently `False` for backward compatibility, but will be switched to `True` in a future version.
super().__init__(use_dict_output=use_dict_output)
```
## Physics
The multicoil physics is defined every easily:
```Python
physics = dinv.physics.MultiCoilMRI(
device=device, noise_model=dinv.physics.GaussianNoise(0.0)
)
```
## Model
For the model we fine-tune a pretrained model ([`deepinv.models.RAM`](https://deepinv.org/api/stubs/deepinv.models.RAM.html.md#deepinv.models.RAM)) [1](#footcite-terris2025reconstruct)
```Python
model = dinv.models.RAM(device=device)
```
## Loss
We define the weighted SSDU loss by first defining the generator that generates the splitting masks.
These splitting masks are multiplied with the measurements as per the original paper.
Finally, the weighted SSDU loss requires knowledge of the original physics generator to define the weight.
#### TIP
Feel free to use any self-supervised loss you like here!
```Python
split_generator = dinv.physics.generator.GaussianMaskGenerator(
(256, 256), acceleration=2, center_fraction=0.0, device=device
)
mask_generator = dinv.physics.generator.MultiplicativeSplittingMaskGenerator(
(256, 256), split_generator, device=device
)
physics_generator = dinv.physics.generator.GaussianMaskGenerator(
(256, 256), acceleration=6, center_fraction=0.04, rng=rng, device=device
)
loss = dinv.loss.mri.WeightedSplittingLoss(
mask_generator=mask_generator, physics_generator=physics_generator
)
```
## Training
We train the model using the self-supervised loss. We randomly split the volume into training slices and validation slices for
early stopping (up to a maximum of 100 epochs).
Because the FastMRI ground truth are cropped magnitude root-sum-of-squares reconstructions, we define a helper metric for evaluation later.
#### HINT
This example trains on GPU to accelerate training in the example.
```Python
def crop(x_net, x):
"""Crop to GT shape then take magnitude."""
return dinv.utils.MRIMixin().rss(
dinv.utils.MRIMixin().crop(x_net, shape=x.shape), multicoil=False
)
class CropPSNR(dinv.metric.PSNR):
def forward(self, x_net=None, x=None, *args, **kwargs):
return super().forward(crop(x_net, x), x, *args, **kwargs)
metric = CropPSNR(max_pixel=None)
train_dataset, val_dataset = torch.utils.data.random_split(
dataset, (0.8, 0.2), generator=rng_cpu
)
trainer = dinv.Trainer(
model=model,
physics=physics,
losses=loss,
metrics=metric,
optimizer=torch.optim.Adam(model.parameters(), lr=1e-6),
train_dataloader=torch.utils.data.DataLoader(
train_dataset, shuffle=True, generator=rng_cpu
),
eval_dataloader=torch.utils.data.DataLoader(val_dataset, generator=rng_cpu),
epochs=0 if str(device) == "cpu" else 100,
save_path=None,
early_stop=3,
early_stop_on_losses=True,
compute_train_metrics=False,
compute_eval_losses=True,
show_progress_bar=False, # disable progress bar for better vis in sphinx gallery.
device=device,
)
model = trainer.train()
model = model.eval()
```
```none
/local/jtachell/deepinv/deepinv/deepinv/training/trainer.py:402: DeprecationWarning: The tuple format for dataset outputs is deprecated and will be removed in a future version.It is recommended to implement your dataset to return a dict with keys 'x', 'y', and 'params' instead of a tuple.
check_dataset(loader.dataset)
/local/jtachell/deepinv/deepinv/deepinv/training/trainer.py:1415: UserWarning: non_blocking_transfers=True but DataLoader.pin_memory=False; set pin_memory=True to overlap host-device copies with compute.
self.setup_train()
The model has 35618813 trainable parameters
/local/jtachell/deepinv/deepinv/deepinv/physics/forward.py:1020: UserWarning: At least one input physics is a DecomposablePhysics, but resulting physics will not be decomposable. `A_dagger` and `prox_l2` will fall back to approximate methods, which may impact performance.
warnings.warn(
/local/jtachell/deepinv/deepinv/deepinv/loss/mri/measplit.py:110: UserWarning: WeightedSplittingLoss detected new y width 213 in forward pass. Recalculating weight (this may take a second)...
warn(
Train epoch 0: TotalLoss=0.276
Eval epoch 0: TotalLoss=0.256, CropPSNR=30.676
Best model saved at epoch 1
Train epoch 1: TotalLoss=0.321
Eval epoch 1: TotalLoss=0.403, CropPSNR=30.72
Best model saved at epoch 2
Train epoch 2: TotalLoss=0.211
Eval epoch 2: TotalLoss=0.296, CropPSNR=30.766
Best model saved at epoch 3
Train epoch 3: TotalLoss=0.265
Eval epoch 3: TotalLoss=0.194, CropPSNR=30.814
Best model saved at epoch 4
Train epoch 4: TotalLoss=0.372
Eval epoch 4: TotalLoss=0.241, CropPSNR=30.844
Best model saved at epoch 5
Train epoch 5: TotalLoss=0.232
Eval epoch 5: TotalLoss=0.421, CropPSNR=30.866
Best model saved at epoch 6
Train epoch 6: TotalLoss=0.232
Eval epoch 6: TotalLoss=0.284, CropPSNR=30.889
Best model saved at epoch 7
Train epoch 7: TotalLoss=0.181
Eval epoch 7: TotalLoss=0.296, CropPSNR=30.915
Best model saved at epoch 8
Early stopping triggered at epoch 7 as validation metrics have not improved in the last 3 validation steps. Disable it with early_stop=None, or modify early_stop>0 to wait for more validation steps.
```
## Evaluation
Now that the model is trained, we test the model on 3 samples
by evaluating the model, plotting and saving the reconstructions and evaluation metrics.
```Python
from torch.utils.data._utils.collate import default_collate
for i in [len(dataset) // 2 - 1, len(dataset) // 2, len(dataset) // 2 + 1]:
# Load slice
x, y, params = default_collate([dataset[i]])
x, y, params = (
x,
y.to(device),
{
k: (v.to(device) if isinstance(v, torch.Tensor) else v)
for (k, v) in params.items()
},
)
physics.update(**params)
# Compute baseline reconstructions
x_adj = physics.A_adjoint(y).detach().cpu()
x_dag = physics.A_dagger(y).detach().cpu()
# Evaluate model
with torch.no_grad():
x_hat = model(y, physics).detach().cpu()
dinv.utils.plot(
{
"GT": x,
"Adjoint": crop(x_adj, x),
"SENSE": crop(x_dag, x),
"Trained": crop(x_hat, x),
},
subtitles=[
"",
f"{metric(x_adj, x).item():.2f} dB",
f"{metric(x_dag, x).item():.2f} dB",
f"{metric(x_hat, x).item():.2f} dB",
],
)
```
* **References:**
* Matthieu Terris, Samuel Hurault, Maxime Song, and Julián Tachella. Reconstruct anything model: a lightweight foundation model for computational imaging. *arXiv preprint arXiv:2503.08915*, 2025.
* Charles Millard and Mark Chiew. A theoretical framework for self-supervised mr image reconstruction using sub-sampling via variable density noisier2noise. *IEEE transactions on computational imaging*, 9:707–720, 2023.
* Burhaneddin Yaman, Seyed Amir Hossein Hosseini, Steen Moeller, Jutta Ellermann, Kâmil Uğurbil, and Mehmet Akçakaya. Self-supervised learning of physics-guided reconstruction neural networks without fully sampled reference data. *Magnetic resonance in medicine*, 84(6):3172–3191, 2020.
* Andrew Wang, Steven McDonagh, and Mike Davies. Benchmarking self-supervised learning methods for accelerated mri reconstruction. *arXiv preprint arXiv:2502.14009*, 2025.
* Dongdong Chen, Julián Tachella, and Mike E Davies. Equivariant imaging: learning beyond the range space. In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, 4379–4388. 2021.
* Florian Knoll, Tullie Murrell, Anuroop Sriram, Nafissa Yakubova, Jure Zbontar, Michael Rabbat, Aaron Defazio, Matthew J Muckley, Daniel K Sodickson, C Lawrence Zitnick, and others. Advancing machine learning for mr image reconstruction with an open competition: overview of the 2019 fastmri challenge. *Magnetic resonance in medicine*, 84(6):3054–3070, 2020.
**Total running time of the script:** (0 minutes 39.077 seconds)
🏷 Tags: MRI
[`Download Jupyter notebook: demo_scan_specific.ipynb`](https://deepinv.org/auto_examples/self-supervised-learning/_downloads/dab47d9d233ee5834e1a29b48217767a/demo_scan_specific.ipynb)
[`Download Python source code: demo_scan_specific.py`](https://deepinv.org/auto_examples/self-supervised-learning/_downloads/7ffb503a30562de1d548f79e11d4acc8/demo_scan_specific.py)
[`Download zipped: demo_scan_specific.zip`](https://deepinv.org/auto_examples/self-supervised-learning/_downloads/8fa417c2f3f659a045b2bfee5aed3ba8/demo_scan_specific.zip)
[Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io)