Patch-based pipeline for 2D histology dataset

[Jarartur]

Hi,
i don't have a lot of experience with torchIO and wanted to ask if the patch-based pipeline described here will work with large 2D images.

Also, if it is possible:
To apply transformations and general preprocessing (like affine aligment) I need to stick with torchIO transformations or can I use something like albumentations (my current pipeline)?

[romainVala]

Hello
yes the patch-based will work, I never test, but I think you just need to put a 1 in the third dimension
(you may get 3D patches, but with a 1 in the third dimension)

I do think you can easily use albumentations, but you should find similar transform in torchio ... are there some missing ?

[romainVala]

this is also the case with torchio
you have to construct a subject with 2 volume in it the input image (type torchio.INTENSITY ) and your target (type torchio.LABEL)
then for a random spatial transform (affine, crop ...) both will have the exact same transform, but for intensity (noise bias gamma ect ...) only the input image will be modify

and note also, that when creating patch, you will get of course the same patch for both

[romainVala]

but I just see, that you want to do a registration task, so it is not that easy than segmentation where your target is a mask of label ...

[Jarartur]

Thank you for providing details on how to do it in torchIO. Still the difference in intensity augmentations won't be ideal.
I will certainly look more into it.

[fepegar]

I'm not sure what's the missing feature. Could you explain a bit more what you mean about the difference in intensity augmentations?

[Jarartur]

I was just referring to romainVala way of making my target something that won't accept intensity augmentations. If I make them 2 torchi.ScalarImages then I think It would be fine.

[fepegar]

Hi, @Jarartur. Why would you like to use TorchIO for 2D images? Understanding your use case will help us suggest solutions.

[romainVala]

why can you not do the same with torchio, instead of torchvision ?

[Jarartur]

One problem in my approach is that I need to make patches from two images that need to be preprocessed based on each other before augmentations and unfortunately, I don't see how I would go about that.
From what I understand I would need to abandon tio.ScalarImage completely and at the same time I can't afford to load all images at once (so I need lazy loading).
Currently I am doing that by writing my custom Dataset from pytorch where I can load and preprocess two images before even going into Dataloader.

[fepegar]

What about using a transform that extracts patches?

import torchio as tio


class SamplePatches:
    def __init__(self, patch_size, num_patches, image_name):
        self.patch_size = patch_size
        self.num_patches = num_patches
        self.image_name = image_name

    def __call__(self, subject):
        sampler = tio.UniformSampler(self.patch_size)
        patches = [p for p in sampler(subject, self.num_patches)]
        first = patches[0]
        for i, p in enumerate(patches[1:]):
            first.add_image(p[self.image_name], f'{self.image_name}_{i + 1}')
        return first


num_patches = 8
patch_size = 64

sheep = tio.datasets.Sheep()
sample_patches = SamplePatches(patch_size, num_patches, 't1')
sheep_patches = sample_patches(sheep)
sheep_patches.plot()

[Jarartur]

Yes I think that's one way to consider. I will try to create something like that but with consideration on the other disscusion.
I am more comfortable now with creating my custom transforms so I think this is the way I will go about doing that.

[romainVala]

May be it is not appropriate (not sure to follow properly here)

But here is an example, with torchio Queue, that perfectly mange the fact to get the exact same patch from 2 image

import torchio as tio
!curl -o image.jpg https://thispersondoesnotexist.com/image
image1 = tio.ScalarImage('image.jpg')
!curl -o image2.jpg https://thispersondoesnotexist.com/image
image2 = tio.ScalarImage('image2.jpg')

subject = tio.Subject(im1=image1, im2=image2)

suj_dataset = tio.SubjectsDataset([subject])
patch_size = 128, 128, 1
sampler = tio.GridSampler(subject, patch_size) #or  tio.UniformSampler(patch_size)

queue = tio.Queue(suj_dataset, sampler=sampler, max_length=64, samples_per_volume=64)

one_patch = queue[10]

#and you can then use the queue as your dataset in a standard torch DataLoader