Resizing

TensorDicom3D.resize_3d[source]

TensorDicom3D.resize_3d(t:TensorMask3D'>), size, scale_factor=None, mode='trilinear', align_corners=True, recompute_scale_factor=None)

A function to resize a 3D image using torch.nn.functional.interpolate

Args:
    t (Tensor): a 3D or 4D Tensor to be resized
    size (int): a tuple with the new x,y,z dimensions of the tensor after resize
    scale_factor, mode, align_corners, recompute_scale_factor: args from F.interpolate
Returns:
    A new `Tensor` with Tensor.size = size

TensorMask3D.resize_3d[source]

TensorMask3D.resize_3d(t:TensorMask3D'>), size, scale_factor=None, mode='trilinear', align_corners=True, recompute_scale_factor=None)

A function to resize a 3D image using torch.nn.functional.interpolate

Args:
    t (Tensor): a 3D or 4D Tensor to be resized
    size (int): a tuple with the new x,y,z dimensions of the tensor after resize
    scale_factor, mode, align_corners, recompute_scale_factor: args from F.interpolate
Returns:
    A new `Tensor` with Tensor.size = size

class Resize3D[source]

Resize3D(size, scale_factor=None, mode='trilinear', align_corners=True, recompute_scale_factor=None, **kwargs) :: RandTransform

A transform that before_call its state at each `__call__`

original = TensorDicom3D.create('../data/series/radiopaedia_10_85902_1.nii.gz')
mask = TensorMask3D.create('../data/masks/radiopaedia_10_85902_1.nii.gz')

original.show()
mask.show(add_to_existing = True, alpha = 0.25, cmap = 'jet')

Resize3D((10,50,50))(original, split_idx = 0).show()
Resize3D((10,50,50))(mask, split_idx = 0).show(add_to_existing = True, alpha = 0.25, cmap = 'jet')

Resampling

resample[source]

resample(img:Image, reference_size, reference_spacing, resample_method, reference_origin=None, reference_direction=None)

Resamples a sitk.Image to an new size, origin, spacing and direction. Usefull when using multiple inputs.

class Resample3D[source]

Resample3D(size:tuple, spacing:tuple, origin:tuple=None, direction:tuple=None, p=1, **kwargs) :: RandTransform

A transform that before_call its state at each `__call__`

Unlike normal images, DICOM images occupy a certain volume of space. The dimension of the voxels therefore says nothing about the real size of the image but only the dimension of the pixels multiplied by the spacing. When processing multiple DICOM sequences at the same time, it is possible that these sequences have a different orientation around space and a different size, although the voxel dimensions are identical. Resample3D allows to adjust the 3D volumes to a uniform size and orientation. This way it can be achieved that the same structure is always present in the same place in different images.

Resample3D((20, 224, 224), (20, 5, 7))(original).show()

Padding

Change size of tensor without changing size of the raw pixel data

TensorDicom3D.pad_to[source]

TensorDicom3D.pad_to(im:TensorMask3D'>), new_size:tuple)

TensorMask3D.pad_to[source]

TensorMask3D.pad_to(im:TensorMask3D'>), new_size:tuple)

class Pad3D[source]

Pad3D(new_size, p=1, **kwargs) :: RandTransform

A transform that before_call its state at each `__call__`

Pad3D((10, 800, 800))(original).show()

Flipping

In medical images, the left and right side often cannot be differentiated from each other (e.g. scans of the head, hand, knee, ...). Therefore, the image orientation is stored in the image header, enabling the viewer system to accurately display the images. For deep learning, only the pixel array is extracted, so the header information is lost. When displaying only the pixel array, the images might already be displayed flipped or in inverted slice order. So, implementing vertical/horizontal flipping as well as flipping alongside the z-axis can be used for data augmentation.

TensorDicom3D.flip_ll_3d[source]

TensorDicom3D.flip_ll_3d(t:TensorMask3D'>))

flips an image laterolateral

TensorMask3D.flip_ll_3d[source]

TensorMask3D.flip_ll_3d(t:TensorMask3D'>))

flips an image laterolateral

TensorDicom3D.flip_ap_3d[source]

TensorDicom3D.flip_ap_3d(t:TensorMask3D'>))

flips an image anterior-posterior

TensorMask3D.flip_ap_3d[source]

TensorMask3D.flip_ap_3d(t:TensorMask3D'>))

flips an image anterior-posterior

TensorDicom3D.flip_cc_3d[source]

TensorDicom3D.flip_cc_3d(t:TensorMask3D'>))

flips an image cranio-caudal

TensorMask3D.flip_cc_3d[source]

TensorMask3D.flip_cc_3d(t:TensorMask3D'>))

flips an image cranio-caudal

class RandomFlip3D[source]

RandomFlip3D(p=0.75, axis=(-1, -2, -3)) :: RandTransform

Randomly flip alongside any axis with probability `p`

torch.stack((original, RandomFlip3D()(original, split_idx = 0), 
                       RandomFlip3D()(original, split_idx = 0), 
                       RandomFlip3D()(original, split_idx = 0))).show(nrow = 15)

Rotating

Medical images should show no rotation, however with removal of the image file header, the pixel array might appear rotated when displayed and therefore be introduced to the model rotated. Fruthermore, in some images the patients might be rotated to some degree. Because of this, rotation of 90° and 180° as well as substeps should be implemented.

TensorDicom3D.rotate_90_3d[source]

TensorDicom3D.rotate_90_3d(t:TensorMask3D'>))

TensorMask3D.rotate_90_3d[source]

TensorMask3D.rotate_90_3d(t:TensorMask3D'>))

TensorDicom3D.rotate_270_3d[source]

TensorDicom3D.rotate_270_3d(t:TensorMask3D'>))

TensorMask3D.rotate_270_3d[source]

TensorMask3D.rotate_270_3d(t:TensorMask3D'>))

TensorDicom3D.rotate_180_3d[source]

TensorDicom3D.rotate_180_3d(t:TensorMask3D'>))

TensorMask3D.rotate_180_3d[source]

TensorMask3D.rotate_180_3d(t:TensorMask3D'>))

class RandomRotate3D[source]

RandomRotate3D(p=0.5) :: RandTransform

Random flip rotates the axial slices of the 3D image 90/180 or 270 degrees with probability `p`

torch.stack((original, RandomRotate3D()(original, split_idx = 0),  
             RandomRotate3D()(original, split_idx = 0),  RandomRotate3D()(original, split_idx = 0))).show(nrow = 15)

Pytorch does not support rotation of 3D images, so some transformations need to be applied slicewise.

TensorDicom3D.apply_along_dim[source]

TensorDicom3D.apply_along_dim(t:TensorMask3D'>), func, dim)

TensorMask3D.apply_along_dim[source]

TensorMask3D.apply_along_dim(t:TensorMask3D'>), func, dim)

class RandomRotate3DBy[source]

RandomRotate3DBy(p=0.5, degrees=(2, 2, 30), axis=[-1, -2, -3]) :: RandTransform

Randomly flip rotates the axial slices of the 3D image 90/180 or 270 degrees with probability `p`

tmp1 = RandomRotate3DBy()(original, split_idx = 0)
tmp2 = RandomRotate3DBy(p=1., degrees=(10, 10, 45),  axis=[-1, -2, -3])(original, split_idx = 0)
original.show(nrow = 15)
tmp1.show(nrow = 15)
tmp2.show(nrow =15)

Rotating by 90 (or 180 and 270) degrees should not be done via RandomRotate3DBy but by rotate_90_3d, as this is approximatly 28 times faster.

"Dihedral" transformation

As the 3D array can be flipped by three sides, but should only be rotated along the z axis, this is not a complete dihedral group. Still multiple combinations of flipping and rotating should be implemented:

1. original (= flipp ll, roate 180 = same as original image)
2. rotate 90
3. rotate 180
4. rotate 270
5. flip ll (=flip ap, rotate 180)
6. flip ap
7. flip cc
8. flip cc, rotate 90
9. flip cc, rotate 180
10. flip cc, rotate 270
11. flip ll, rotate 90
12. flipp ll, rotate 270
13. flip ap, rotate 90
14. flip ap rotate 270
15. flip cc, flip ll, rotate 90
16. flip cc, flipp ll, rotate 270
17. flip cc, flip ap, rotate 90
18. flip cc, flip ap rotate 270

I am not sure if this is complete...

TensorDicom3D.dihedral3d[source]

TensorDicom3D.dihedral3d(x:TensorMask3D'>), k)

apply dihedral transforamtions to the 3D Dicom Tensor

TensorMask3D.dihedral3d[source]

TensorMask3D.dihedral3d(x:TensorMask3D'>), k)

apply dihedral transforamtions to the 3D Dicom Tensor

class RandomDihedral3D[source]

RandomDihedral3D(p=1.0, nm=None, before_call=None, **kwargs) :: RandTransform

randomly flip and rotate the 3D Dicom volume with a probability of `p`

dihedral = RandomDihedral3D()
torch.stack((original, dihedral(original, split_idx = 0), dihedral(original, split_idx = 0), 
                 dihedral(original, split_idx = 0),dihedral(original, split_idx = 0), 
                 dihedral(original, split_idx = 0))).show(nrow=15)

Random crop

A reasonable approach for 3D medical images would be a presizing to a uniform but large volume and subsequent random cropping to the target dimension. As most areas of interest are located centrally in the image/volume, some cropping can always be applied.
Also random cropping should be applied after any rotation, that is not in 90/180/270 degrees, so that empty margins are cropped.

TensorDicom3D.crop_3d[source]

TensorDicom3D.crop_3d(t:TensorMask3D'>), crop_by:(<class 'int'>, <class 'float'>), perc_crop=False)

Similar to function `crop_3d_tensor`, but no checking for margin formats is done, as they were correctly passed to this function by RandomCrop3D.encodes

TensorMask3D.crop_3d[source]

TensorMask3D.crop_3d(t:TensorMask3D'>), crop_by:(<class 'int'>, <class 'float'>), perc_crop=False)

Similar to function `crop_3d_tensor`, but no checking for margin formats is done, as they were correctly passed to this function by RandomCrop3D.encodes

class RandomCrop3D[source]

RandomCrop3D(crop_by, rand_crop_xyz, perc_crop=False, p=1, **kwargs) :: RandTransform

Randomly crop the 3D volume with a probability of `p`
The x axis is the "slice" axis, where no cropping should be done by default
During inference, the behaviour is switched to center-crop without random variation

Args
    crop_by: number of pixels or pecantage of pixel to be removed at each side. E.g. if (0, 5, 5), 0 pixel in the x axis, but 10 pixels in eacht y and z axis will be cropped (5 each side)
    rand_crop_xyz: range in which the cropping window is allowed to vary.
    perc_crop: if true, no absolute but relative number of pixels are cropped

Crop = RandomCrop3D((10,50,50), (10,20,20), False)

torch.stack((Crop(original, split_idx = 0), Crop(original, split_idx = 0), 
             Crop(original, split_idx = 0), Crop(original, split_idx = 0))).show(nrow = 10)
im = Crop(original).resize_3d((10, 100, 100))

class ResizeCrop3D[source]

ResizeCrop3D(crop_by, resize_to, perc_crop=False, p=1, **kwargs) :: RandTransform

A transform that before_call its state at each `__call__`

Mask erasing

Sometimes, the images are in an uniform format and the area of interest can be expected in a certain image region. A mask, which covers the areas of interest can help to futher reduce the image volume.

crop_mask = TensorMask3D(torch.ones(4, 100, 20)).pad_to((10, 100, 100))
crop_mask = crop_mask + crop_mask.rotate_90_3d()
crop_mask = torch.where(crop_mask == 0, 0, 1)

crop_mask2 = TensorMask3D(torch.ones(10, 100, 20)).pad_to((10, 100, 100))
crop_mask2 = crop_mask2 + crop_mask2.rotate_90_3d()
crop_mask2 = torch.where(crop_mask2 == 0, 1, 0)

crop_mask.show()
crop_mask2.show()

class MaskErease[source]

MaskErease(mask, pad=None, p=1.0) :: DisplayedTransform

ereases image areas in dependence of a mask. Strips black spaces afterwards

MaskErease(mask = crop_mask)(im).show()
MaskErease(mask = crop_mask2)(im).show()

Random change of perspective

im2 = TensorDicom3D.create('../data/example_grid.nii.gz')
im2 = im2.unsqueeze(0)
im2.show()

class RandomPerspective3D[source]

RandomPerspective3D(input_size, p=0.5, distortion_scale=0.25) :: RandTransform

A transform that before_call its state at each `__call__`

RandomPerspective3D(im.size(-1), p = 1.)(im2, split_idx=0).show()

Further perspective distortions

Augmentations based on Grid resampling. Not as many possibilities as RandomPerspective, but does allow to fill empty areas with mirrored image and not just 0 padding.

Perspective warping

Light warping can mimic artifacts in MRI images or breathing artifacts in CT images

warp_3d[source]

warp_3d(h, w, magnitude_h, magnitude_w)

TensorDicom3D.grid_tfms[source]

TensorDicom3D.grid_tfms(t:TensorMask3D'>), func, mode)

wrapper for grid tfms

TensorMask3D.grid_tfms[source]

TensorMask3D.grid_tfms(t:TensorMask3D'>), func, mode)

wrapper for grid tfms

class RandomWarp3D[source]

RandomWarp3D(p=0.5, max_magnitude=0.25) :: RandTransform

A transform that before_call its state at each `__call__`

RandomWarp3D(p=1, max_magnitude=0.5)(im2, split_idx = 0).show()
RandomWarp3D(p=1, max_magnitude=0.5)(im2, split_idx = 0).show()
RandomWarp3D(p=1, max_magnitude=0.5)(im2, split_idx = 0).show()

Sheering

sheer_3d[source]

sheer_3d(h, w, magnitude_h, magnitude_w)

applies a random sheer to the tensor

class RandomSheer3D[source]

RandomSheer3D(p=0.5, max_magnitude=0.25) :: RandomWarp3D

A transform that before_call its state at each `__call__`

RandomSheer3D(p=1, max_magnitude=0.5)(im2, split_idx = 0).show()
RandomSheer3D(p=1, max_magnitude=0.5)(im2, split_idx = 0).show()
RandomSheer3D(p=1, max_magnitude=0.5)(im2, split_idx = 0).show()

Trapezoid

trapezoid_3d[source]

trapezoid_3d(h, w, magnitude)

applies a random sheer to the tenspr

class RandomTrapezoid3D[source]

RandomTrapezoid3D(p=0.5, max_magnitude=0.25) :: RandomWarp3D

A transform that before_call its state at each `__call__`

RandomTrapezoid3D(p=1, max_magnitude=0.5)(im2, split_idx = 0).show()
RandomTrapezoid3D(p=1, max_magnitude=0.5)(im2, split_idx = 0).show()
RandomTrapezoid3D(p=1, max_magnitude=0.5)(im2, split_idx = 0).show()

Random Gaussian noise

Older scanners with lower field strength (MRI), fewer slices (CT) and/or older algorithms can be more noisy. So adding some random noise to the data, could improve model performance.

TensorDicom3D.gaussian_noise[source]

TensorDicom3D.gaussian_noise(t:TensorDicom3D, std)

class RandomNoise3D[source]

RandomNoise3D(p=0.5, std_range=[0.01, 0.1]) :: RandTransform

A transform that before_call its state at each `__call__`

Noise= RandomNoise3D(p=1)
RandomNoise3D(p=1)(im.mean_scale(), split_idx=0).show()
RandomNoise3D(p=1)(im.mean_scale(), split_idx=0).show()
RandomNoise3D(p=1)(im.mean_scale(), split_idx=0).show()
RandomNoise3D(p=1)(im.mean_scale(), split_idx=0).show()

Gaussian blur

class RandomBlur3D[source]

RandomBlur3D(p=0.5, kernel_size_range=[5, 11], sigma=0.5) :: RandTransform

A transform that before_call its state at each `__call__`

RandomBlur3D(p=1., sigma = 10)(im, split_idx=0).show()

Lightning transforms

Simple brightness and contrast controlls can be dona via a linear function:

x = alpha * x_i + beta  

Here x_i is the respective pixel, alpha allows simple contrast control, beta allows simple brightness control.

TensorDicom3D.rescale[source]

TensorDicom3D.rescale(t:TensorDicom3D, new_min=0, new_max=1)

TensorDicom3D.adjust_brightness[source]

TensorDicom3D.adjust_brightness(x:TensorDicom3D, beta)

class RandomBrightness3D[source]

RandomBrightness3D(p=0.5, beta_range=[-0.3, 0.3]) :: RandTransform

A transform that before_call its state at each `__call__`

torch.stack((im.mean_scale(), 
             RandomBrightness3D(p=1., beta_range=[0.9, 1])(im.mean_scale(), split_idx = 0), 
             RandomBrightness3D(p=1., beta_range=[-0.9, -1])(im.mean_scale(), split_idx = 0))).show()

Contrast

TensorDicom3D.adjust_contrast[source]

TensorDicom3D.adjust_contrast(x:TensorDicom3D, alpha)

class RandomContrast3D[source]

RandomContrast3D(p=0.6, alpha_range=[0.7, 2.0]) :: RandTransform

A transform that before_call its state at each `__call__`

im.mean_scale().show()
RandomContrast3D(p=1.)(im.mean_scale(), split_idx = 0).show()
RandomContrast3D(p=1.)(im.mean_scale(), split_idx = 0).show()

Elastic transforms

https://theaisummer.com/medical-image-processing/#8-intensity-normalization-in-medical-images

def elastic_transform_3d(image, labels=None, alpha=4, sigma=35, bg_val=0.1):
    """
    Elastic deformation of images as described in
    Simard, Steinkraus and Platt, "Best Practices for
    Convolutional Neural Networks applied to Visual
    Document Analysis", in
    Proc. of the International Conference on Document Analysis and
    Recognition, 2003. 

    Modified from:
    https://gist.github.com/chsasank/4d8f68caf01f041a6453e67fb30f8f5a
    https://github.com/fcalvet/image_tools/blob/master/image_augmentation.py#L62

    Modified to take 3D inputs
    Deforms both the image and corresponding label file
    image linear/trilinear interpolated

    Label volumes nearest neighbour interpolated
    """
    assert image.ndim == 3
    shape = image.shape
    dtype = image.dtype

    # Define coordinate system
    coords = np.arange(shape[0]), np.arange(shape[1]), np.arange(shape[2])

    # Initialize interpolators
    im_intrps = RegularGridInterpolator(coords, image,
                                                method="linear",
                                                bounds_error=False,
                                                fill_value=bg_val)

    # Get random elastic deformations
    dx = gaussian_filter((np.random.rand(*shape) * 2 - 1), sigma,
                         mode="constant", cval=0.) * alpha
    dy = gaussian_filter((np.random.rand(*shape) * 2 - 1), sigma,
                         mode="constant", cval=0.) * alpha
    dz = gaussian_filter((np.random.rand(*shape) * 2 - 1), sigma,
                        mode="constant", cval=0.) * alpha

    # Define sample points
    x, y, z = np.mgrid[0:shape[0], 0:shape[1], 0:shape[2]]
    indices = np.reshape(x + dx, (-1, 1)), \
             np.reshape(y + dy, (-1, 1)), \
             np.reshape(z + dz, (-1, 1))

    # Interpolate 3D image image
    image = np.empty(shape=image.shape, dtype=dtype)
    image = im_intrps(indices).reshape(shape)

    # Interpolate labels
    if labels is not None:
        lab_intrp = RegularGridInterpolator(coords, labels,
                                           method="nearest",
                                           bounds_error=False,
                                           fill_value=0)

        labels = lab_intrp(indices).reshape(shape).astype(labels.dtype)
        return image, labels

    return image

Putting it all together

A good workflow would be to apply random crop to all images after one transformation. For this, the images should be presized to a size, just some pixels larger than desired, then transformed and cropped to the final size. Using this approach empty space, which e.g. appears after RandomRotate3DBy will be cropped and will not affect the accuracy of the model. One only has to be careful, that the region of interest, e.g. the prostate, will be in every cropped image.

Crop = RandomCrop3D((2,10,10), (1,2,2))

tfms = [RandomBrightness3D(), RandomContrast3D(), RandomWarp3D(), RandomDihedral3D(), RandomNoise3D(), RandomRotate3DBy()]
tfms = [Pipeline([RandomBrightness3D(p=1.), Crop], split_idx = 0), 
        Pipeline([RandomContrast3D(p=1.), Crop], split_idx = 0), 
        Pipeline([RandomWarp3D(p=1.), Crop], split_idx = 0), 
        Pipeline([RandomDihedral3D(p=1.), Crop], split_idx = 0), 
        Pipeline([RandomNoise3D(p=1.), Crop], split_idx = 0), 
        Pipeline([RandomRotate3DBy(p=1.), Crop], split_idx = 0)]

comp = setup_aug_tfms(tfms)
comp

[Pipeline: RandomBrightness3D -- {'p': 1.0} -> RandomCrop3D -- {'p': 1},
 Pipeline: RandomContrast3D -- {'p': 1.0} -> RandomCrop3D -- {'p': 1},
 Pipeline: RandomWarp3D -- {'p': 1.0} -> RandomCrop3D -- {'p': 1},
 Pipeline: RandomDihedral3D -- {'p': 1.0} -> RandomCrop3D -- {'p': 1},
 Pipeline: RandomNoise3D -- {'p': 1.0} -> RandomCrop3D -- {'p': 1},
 Pipeline: RandomRotate3DBy -- {'p': 1.0, 'degrees': (2, 2, 30), 'axis': [-1, -2, -3]} -> RandomCrop3D -- {'p': 1}]

ims = [t(im).squeeze() for t in tfms]
torch.stack(ims).show(nrow = 6)

Creating a pseudo-color channel

Pytorch expects the images in the following format:

B C D H W

Here:

• B = Batch dimension
• C = Number of Channels (e.g. color)
• D = Depth of the image (= number of slices)
• H = Height of the image
• W = Width of the image

@patch
def make_pseudo_color(t: (TensorDicom3D, TensorMask3D)): 
    '''
    The 3D CNN still expects color images, so a pseudo color image needs to be created as long as I don't adapt the 3D CNN
    '''
    if t.ndim == 3:
        return t.unsqueeze(0).float()  
    elif t.ndim == 4:
        return t.unsqueeze(1).float()
    else: 
        return t  

class PseudoColor(RandTransform):
    split_idx, p = None, 1
    
    def __init__(self, p=1): 
        super().__init__(p=p)

    def __call__(self, b, split_idx=None, **kwargs):
        "change in __call__ to enforce, that the Transform is always applied on every dataset. "
        return super().__call__(b, split_idx=split_idx, **kwargs) 
    
    def encodes(self, x:(TensorDicom3D, TensorMask3D)): 
        return x.make_pseudo_color()

MakeColor = PseudoColor()
im.shape, MakeColor(im, split_idx = 0).shape

((10, 224, 224), (1, 10, 224, 224))

aug_transforms_3d[source]

aug_transforms_3d(p_all=0.1, warp=True, p_warp=None, sheer=True, p_sheer=None, trapezoid=True, p_trapezoid=None, dihedral=True, p_dihedral=None, brightness=True, p_brightness=None, contrast=True, p_contrast=None, noise=True, p_noise=None, rotate_by=True, p_rotate_by=None, flip=True, p_flip=None, rotate=True, p_rotate=None, blur=True, p_blur=None, crop=True, p_crop=1)

tmp = Pipeline(aug_transforms_3d(p_all = 1.), split_idx=0)(im)
print(tmp.size())
tmp.show()

(10, 224, 224)

Mask transformations

Transformations for the mask in segmentation tasks. If it is a multilabel segmentation task, the mask needs to be converted into a one hot encoded tensor.

TensorMask3D.clamp_to_range[source]

TensorMask3D.clamp_to_range(x:TensorMask3D, lwr, upr)

class ClampMask3D[source]

ClampMask3D(lwr=0, upr=1, p=1) :: RandTransform

Clamps/Clips mask value to a range

TensorMask3D.reduce_classes[source]

TensorMask3D.reduce_classes(x:TensorMask3D, classes:(<class 'list'>, <class 'tuple'>))

class ReduceClasses[source]

ReduceClasses(reduce_to, p=1) :: RandTransform

Removes classes from mask

mask.reduce_classes([1]).unique()

TensorMask3D([0., 1.])