Convolutional neural networks for semantic segmentation of FIB-SEM volumetric image data

Abstract

Focused ion beam scanning electron microscopy (FIB-SEM) is a well-established microscopy technique for 3D imaging of porous materials. We investigate three porous samples of ethyl cellulose microporous films made from ethyl cellulose and hydroxypropyl cellulose (EC/HPC) polymer blends. These types of polymer blends are used as coating materials on various pharmaceutical tablets or pellets and form a continuous network of pores in the film. Understanding the microstructures of these porous networks allow for controlling drug release. We perform semantic segmentation of the image data, separating the solid parts of the material from the pores to accurately quantify the microstructures in terms of porosity. Segmentation of FIB-SEM data is complicated because in each 2D slice there is 2.5D information, due to parts of deeper underlying cross-sections shining through in porous areas. The supposed shine-through effect greatly complicates the segmentation in regards to two factors; uncertainty in the positioning of the microstructural features and overlapping grayscale intensities between pore and solid regions. In this work, we explore different convolutional neural networks (CNNs) for pixelwise classification of FIB-SEM data, where the class of each pixel is predicted using a three-dimensional neighborhood of size (nx; ny; nz). In total, we investigate six types of CNN architectures with different hyperparameters, dimensionalities, and inputs. For assessing the classification performance we consider the mean intersection over union (mIoU), also called Jaccard index. All the investigated CNNs are well suited to the problem and perform good segmentations of the FIB-SEM data. The so-called standard 2DCNN performs the best overall followed by different varieties of 2D and 3D CNN architectures. The best performing models utilize larger neighborhoods, and there is a clear trend that larger neighborhoods boost performance. Our proposed method improves results on all metrics by 1.35 - 3.14 % compared to a previously developed method for the same data using Gaussian scale-space features and a random forest classifier. The porosities for the three HPC samples are estimated to 20.34, 33.51, and 45.75 %, which is in close agreement with the expected porosities of 22, 30, and 45 %. Interesting future work would be to let multiple experts segment the same image to obtain more accurate ground truths, to investigate loss functions that better correlate with the porosity, and to consider other neighborhood sizes. Ensemble learning methods could potentially boost results even further, by utilizing multiple CNNs and/or other machine learning models together.