Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Mar;26(3):412-423.
doi: 10.1016/j.acra.2018.08.003. Epub 2018 Sep 6.

Convolutional Neural Networks with Template-Based Data Augmentation for Functional Lung Image Quantification

Nicholas J Tustison  1 , Brian B Avants  2 , Zixuan Lin  3 , Xue Feng  3 , Nicholas Cullen  4 , Jaime F Mata  3 , Lucia Flors  5 , James C Gee  4 , Talissa A Altes  5 , John P Mugler Iii  3 , Kun Qing  3
Affiliations

Convolutional Neural Networks with Template-Based Data Augmentation for Functional Lung Image Quantification

Nicholas J Tustison et al. Acad Radiol. 2019 Mar.

Abstract

Rationale and objectives: We propose an automated segmentation pipeline based on deep learning for proton lung MRI segmentation and ventilation-based quantification which improves on our previously reported methodologies in terms of computational efficiency while demonstrating accuracy and robustness. The large data requirement for the proposed framework is made possible by a novel template-based data augmentation strategy. Supporting this work is the open-source ANTsRNet-a growing repository of well-known deep learning architectures first introduced here.

Materials and methods: Deep convolutional neural network (CNN) models were constructed and trained using a custom multilabel Dice metric loss function and a novel template-based data augmentation strategy. Training (including template generation and data augmentation) employed 205 proton MR images and 73 functional lung MRI. Evaluation was performed using data sets of size 63 and 40 images, respectively.

Results: Accuracy for CNN-based proton lung MRI segmentation (in terms of Dice overlap) was left lung: 0.93 ± 0.03, right lung: 0.94 ± 0.02, and whole lung: 0.94 ± 0.02. Although slightly less accurate than our previously reported joint label fusion approach (left lung: 0.95 ± 0.02, right lung: 0.96 ± 0.01, and whole lung: 0.96 ± 0.01), processing time is <1 second per subject for the proposed approach versus ∼30 minutes per subject using joint label fusion. Accuracy for quantifying ventilation defects was determined based on a consensus labeling where average accuracy (Dice multilabel overlap of ventilation defect regions plus normal region) was 0.94 for the CNN method; 0.92 for our previously reported method; and 0.90, 0.92, and 0.94 for expert readers.

Conclusion: The proposed framework yields accurate automated quantification in near real time. CNNs drastically reduce processing time after offline model construction and demonstrate significant future potential for facilitating quantitative analysis of functional lung MRI.

Keywords: ANTsRNet; Advanced Normalization Tools; Hyperpolarized gas imaging; Neural networks; Proton lung MRI; U-net.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Illustration of the proposed workflow. Training the U-net models for both proton and ventilation imaging includes template-based data augmentation. This offline training is computationally intensive but is only performed once. Subsequent individual subject preprocessing includes MR denoising and bias correction. The proton mask determined from the proton U-net model is included as a separate channel (in deep learning software parlance) for ventilation image processing.
Figure 2.
Figure 2.
Side-by-side image comparison showing the effects of preprocessing on the proton (top) and ventilation (bottom) MRI. (a) Uncorrected image showing MR field inhomogeneity and noise. (b) Corresponding corrected image in which the bias effects have been ameliorated.
Figure 3.
Figure 3.
The modified U-net architecture for both structural and functional lung segmentation (although certain parameters, specifically the number of filters per convolution layer, are specific to the functional case). Network layers are represented as boxes with arrows designating connections between layers. The main parameter value for each layer is provided above the corresponding box. Each layer of the descending (or “encoding”) branch of the network is characterized by two convolutional layers. Modification of the original architecture includes an intermediate dropout layer for regularization (dropout rate = 0.2). A max pooling operation produces the feature map for the next series. The ascending (or “decoding”) branch is similarly characterized. A convolutional transpose operation is used to upsample the feature map following a convolution→ dropout→ convolution layer series until the final convolutional operation which yields the segmentation probability maps.
Figure 4.
Figure 4.
Template-based data augmentation for the proton (left) and ventilation (right) U-net model generation. For both cases, a template is created, or selected, to generate the transforms to and from the template. The derived deformable, invertible transform for the kth subject, Sk to the template, T, is denoted by φk: Sk↔T. These subject-specific mappings are used during model training (but not the template itself). Data augmentation occurs by randomly choosing a reference subject and a target subject during batch processing. In the illustration above, the sample mapping of Subject 1 to the space of Subject 2, represented by the green curved arrow, is defined as φ2−1(φ1).
Figure 5.
Figure 5.
The Dice overlap coefficient for the left and right lungs (and their combination) between the updated latter requires significantly less computation time.
Figure 6.
Figure 6.
The Dice overlap coefficient for total, normal lung, and ventilation defect regions for segmentation of the functional evaluation data set.
Figure 7.
Figure 7.
Problematic case showing potential issues with the JLF approach (left) for proton lung segmentation where a difficult pairwise image registration caused segmentation failure. In contrast, by learning features directly, the U-net approach (right) avoids possible registration difficulties.
Figure 8.
Figure 8.
Ventilation segmentation comparison between a human reader and the two computational approaches. Notice the effects of the partial voluming at the apex of the lungs, indicated by the yellow arrow, which are labeled as ventilation defect by the Atropos approach whereas U-net and the human reader correctly label this region. (Color version of figure is available online.)
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Liu Z, Araki T, Okajima Y, et al. Pulmonary hyperpolarized noble gas MRI: recent advances and perspectives in clinical application. Eur J Radiol 2014; 83:1282–1291. doi:10.1016/j.ejrad.2014.04.014. - DOI - PubMed
    1. Roos JE, McAdams HP, Kaushik SS, et al. Hyperpolarized gas MR imaging: technique and applications. Magn Reson Imaging Clin N Am 2015; 23:217–229. doi:10.1016/j.mric.2015.01.003. - DOI - PMC - PubMed
    1. Adamson EB, Ludwig KD, Mummy DG, et al. Magnetic resonance imaging with hyperpolarized agents: methods and applications. Phys Med Biol 2017; 62:R81–R123. doi:10.1088/1361-6560/aa6be8. - DOI - PMC - PubMed
    1. Svenningsen S, Kirby M, Starr D, et al. What are ventilation defects in asthma? Thorax 2014; 69:63–71. doi:10.1136/thoraxjnl-2013-203711. - DOI - PubMed
    1. Tustison NJ, Altes TA, Song G, et al. Feature analysis of hyperpolarized helium-3 pulmonary MRI: a study of asthmatics versus nonasthmatics. Magn Reson Med 2010; 63:1448–1455. doi:10.1002/mrm.22390. - DOI - PubMed

Publication types