Why a new class?

Fastai 2 provides support to read and display medical images using pydicom and pillow; however, only 2D images can be read. Therefore, no direct support of 3D volumes is provided by fastai. Furthermore, fastai only allows to read DICOM 2D images, not volumes, DICOM series or any of the other medical formats such as NIfTI, NRRD etc. For effective work with medical data, all possible formats should be supported. Therefore, in faimed3d, SimpleITK is used to read images. SimpleITK is a powerfull library which can handle many data formats, including all of the above mentioned and many more. It is widely used for medical applications and very fast, as it is written in pure C++.
To allow specific augmentations, only to 3D data, two new classes are used in faimed3d: TensorDicom3D for 3D image volumes and TensorMask3D for segmentation masks. Both are subclasses to torch.Tensor. A basic support for header information will be implemented in both classes, as the header sometimes has important information, e.g. for re-scaling of the pixel values, pixel spacing or patient orientation.

Reading medical files with TensorDicom3D

class TensorDicom3D[source]

TensorDicom3D(x, metadata=None, *args, **kwargs) :: TensorBase

Base class for 3D medical images

class TensorMask3D[source]

TensorMask3D(x, metadata=None, *args, **kwargs) :: TensorDicom3D

Base class for 3d segmentation, inherits from TensorDicom3D

fn = '../data/series/radiopaedia_10_85902_1.nii.gz'
im = TensorDicom3D.create(fn, load_header=True)

test_eq(hasattr(im, '_metadata'), True)
test_eq(hasattr(im[0], '_metadata'), True)
im2 = im
test_eq(im._metadata, im2._metadata)
test_eq(isinstance(im[im > 3], Tensor), True)
_ = torch.stack((im, im2))

Handlers for metadata

Some metadata needs to be modified, acessed more than other, so some wrapper functions are supplied.

TensorDicom3D.set_spacing[source]

TensorDicom3D.set_spacing(t:TensorDicom3D, spacing)

TensorDicom3D.get_spacing[source]

TensorDicom3D.get_spacing(t:TensorDicom3D)

TensorDicom3D.set_origin[source]

TensorDicom3D.set_origin(t:TensorDicom3D, origin)

TensorDicom3D.get_origin[source]

TensorDicom3D.get_origin(t:TensorDicom3D)

TensorDicom3D.set_direction[source]

TensorDicom3D.set_direction(t:TensorDicom3D, direction)

TensorDicom3D.get_direction[source]

TensorDicom3D.get_direction(t:TensorDicom3D)

Display 3D images

Function to display the 3D volume in axial, coronal or sagittal reconstruction. The display functions does not consider the spacing between pixels, so reconstructions may look unusual.

show_image_3d[source]

show_image_3d(t:Tensor'>), axis:int=0, figsize:int=(15, 15), cmap:str='bone', nrow:int=10, alpha=1.0, return_grid=False, add_to_existing=False, **kwargs)

Plots 2D slices of a 3D image alongside a prior specified axis.
Args:
    t: a 3D numpy.ndarray or torch.Tensor
    axis: axis to split 3D array to 2D images
    figsize, cmap: passed to plt.imshow
    nrow: passed to torchvision.utils.make_grid
    return_grid: Whether the grid should be returned for further processing or if the plot should be displayed.
    add_to_existing: if set to true, no new figure will be created. Used for mask overlays

show_image_3d(im, axis = 0, nrow = 10)

Sometimes multiple 3D images (e.g. a batch) need to be displayed. With a wrapper for show_image_3d this is possible.

show_images_3d[source]

show_images_3d(t:Tensor, axis:int=0, figsize:int=(15, 15), cmap:str='bone', nrow:int=10, alpha=1.0, return_grid=False, add_to_existing=False, **kwargs)

Displays multiple 3D images (e.g. a batch) by flattening the 4th dimension of the tensor
and then calling show_image_3d
Args:
    t (torch.Tensor): input image with dimension B x D x H x W or B x C x D x H x W,
                      however C is ignored for display
    axis (int): Axis to split images to multiple 2D slices. `show_images_3d` does not
                know about pixel spacing, so choosing a non-standard axis will result in
                atypical looking images.
    figsize, cmap, nrow, alpha: passed to plot function
    return_grid (bool): return the grid, not the plot for further processing
    add_to_existing (bool): adds plot to existing plot, not calling plot.new.
                            Usefull for overlays, e.g. with masks.

show_images_3d(torch.stack((im,im)), axis = 0, nrow = 31, figsize = (25, 15))

from torch import Tensor # for compatibility with show_docs

TensorDicom3D.show[source]

TensorDicom3D.show(t:TensorDicom3D, axis:int=0, figsize:int=(15, 15), cmap:str='bone', nrow:int=10, **kwargs)

displays the 3D image as a mosaik

im.show()

Rendering 3D objects

Somtimes the mask is better viewed as a 3D object. Rendering is implemented as described in this example: https://scikit-image.org/docs/dev/auto_examples/edges/plot_marching_cubes.html A faster, more flexible way might be using ipyvolume or vtk.

To implement 3D rendering for the mask, the TensorMask3D class needs to be expanded.

TensorDicom3D.strip[source]

TensorDicom3D.strip(x:TensorDicom3D)

class TensorMask3D[source]

TensorMask3D(x, metadata=None, *args, **kwargs) :: TensorDicom3D

Base class for 3d segmentation, inherits from TensorDicom3D

im  = TensorMask3D.create('../data/masks/radiopaedia_10_85902_1.nii.gz')       
im.render_3d(alpha = (0.15,))

im.calc_volume()

{'background': 5479506.0, 'total_mask_volume': 32997.8452155143, 'class 1': 32997.97265625}