Advertisement

Validation of an established deep learning auto-segmentation tool for cardiac substructures in 4D radiotherapy planning scans

Open AccessPublished:August 02, 2022DOI:https://doi.org/10.1016/j.phro.2022.07.003

      Highlights

      • Cardiotoxicity is a common complication of lung cancer radiotherapy.
      • Segmentation of cardiac substructures is time-consuming and challenging.
      • Deep learning segmentation tools can perform this task in 3D and 4D scans.
      • Performance is high when assessed geometrically, dosimetrically and clinically.
      • Auto-segmentation tools may accelerate clinical workflows and enable research.

      Abstract

      Background

      Emerging data suggest that dose-sparing several key cardiac regions is prognostically beneficial in lung cancer radiotherapy. The cardiac substructures are challenging to contour due to their complex geometry, poor soft tissue definition on computed tomography (CT) and cardiorespiratory motion artefact. A neural network was previously trained to generate the cardiac substructures using three-dimensional radiotherapy planning CT scans (3D-CT). In this study, the performance of that tool on the average intensity projection from four-dimensional (4D) CT scans (4D-AVE), now commonly used in lung radiotherapy, was evaluated.

      Materials and Methods

      The 4D-AVE of n=20 patients completing radiotherapy for lung cancer 2015–2020 underwent manual and automated cardiac substructure segmentation. Manual and automated substructures were compared geometrically and dosimetrically. Two senior clinicians also qualitatively assessed the auto-segmentation tool’s output.

      Results

      Geometric comparison of the automated and manual segmentations exhibited high levels of similarity across parameters, including volume difference (11.8% overall) and Dice similarity coefficient (0.85 overall), and were consistent with 3D-CT performance. Differences in mean (median 0.2 Gy, range −1.6–0.3 Gy) and maximum (median 0.4 Gy, range −2.2–0.9 Gy) doses to substructures were generally small. Nearly all structures (99.5 %) were deemed to be appropriate for clinical use without further editing.

      Conclusions

      Cardiac substructure auto-segmentation using a deep learning-based tool trained on a 3D-CT dataset was feasible on the 4D-AVE scan, meaning this tool is suitable for use on 4D-CT radiotherapy planning scans. Application of this tool would increase the practicality of routine clinical cardiac substructure delineation, and enable further cardiac radiation effects research.

      Keywords

      1. Introduction

      Segmentation of tumors and organs-at-risk (OARs) is a pivotal step in contemporary, inverse-planned radiotherapy (RT). Prospectively defining clinical dose goals for these structures, allows tumor control probability to be maximised while normal tissue complication probability is minimised [
      • Teoh S.
      • Fiorini F.
      • George B.
      • Vallis K.A.
      • Van den Heuvel F.
      Proton vs photon: a model-based approach to patient selection for reduction of cardiac toxicity in locally advanced lung cancer.
      ,
      • Thor M.
      • Deasy J.
      • Iyer A.
      • Bendau E.
      • Fontanella A.
      • Apte A.
      • et al.
      Toward personalized dose-prescription in locally advanced non-small cell lung cancer: validation of published normal tissue complication probability models.
      ]. The delineation of structures on planning scans is both time-consuming [
      • Thind K.
      • Roumeliotis M.
      • Mann T.
      • Van Dyke L.
      • Martell K.
      • Smith W.
      • et al.
      Increasing demand on human capital and resource utilization in radiation therapy: the past decade.
      ] and prone to inter-operator variability [
      • Vinod S.K.
      • Jameson M.G.
      • Min M.
      • Holloway L.C.
      Uncertainties in volume delineation in radiation oncology: a systematic review and recommendations for future studies.
      ], with the latter resulting in negative clinical impact [
      • Jameson M.G.
      • Kumar S.
      • Vinod S.K.
      • Metcalfe P.E.
      • Holloway L.C.
      Correlation of contouring variation with modeled outcome for conformal non-small cell lung cancer radiotherapy.
      ]. Overcoming these drawbacks, automated segmentation has been an interdisciplinary research focus, with tools now available for several primary tumors [
      • Kumar V.
      • Gu Y.
      • Basu S.
      • Berglund A.
      • Eschrich S.A.
      • Schabath M.B.
      • et al.
      QIN “radiomics: the process and the challenges”.
      ] and associated regional OARs [
      • Brouwer C.L.
      • Boukerroui D.
      • Oliveira J.
      • Looney P.
      • Steenbakkers R.J.H.M.
      • Langendijk J.A.
      • et al.
      Assessment of manual adjustment performed in clinical practice following deep learning contouring for head and neck organs at risk in radiotherapy.
      ,
      • Mohammadi R.
      • Shokatian I.
      • Salehi M.
      • Arabi H.
      • Shiri I.
      • Zaidi H.
      Deep learning-based auto-segmentation of organs at risk in high-dose rate brachytherapy of cervical cancer.
      ].
      Incidental cardiac radiation dose correlates negatively with survival in lung cancer [
      • Atkins K.M.
      • Rawal B.
      • Chaunzwa T.L.
      • Lamba N.
      • Bitterman D.S.
      • Williams C.L.
      • et al.
      Cardiac radiation dose, cardiac disease, and mortality in patients with lung cancer.
      ,
      • Speirs C.K.
      • DeWees T.A.
      • Rehman S.
      • Molotievschi A.
      • Velez M.A.
      • Mullen D.
      • et al.
      Heart dose is an independent dosimetric predictor of overall survival in locally advanced non-small cell lung cancer.
      ,
      • Vivekanandan S.
      • Fenwick J.D.
      • Counsell N.
      • Panakis N.
      • Stuart R.
      • Higgins G.S.
      • et al.
      Associations between cardiac irradiation and survival in patients with non-small cell lung cancer: validation and new discoveries in an independent dataset.
      ,
      • McWilliam A.
      • Kennedy J.
      • Hodgson C.
      • Vasquez E.
      • Faivre-Finn C.
      • Van H.M.
      Radiation dose to heart base linked with poorer survival in lung cancer patients.
      ] and particular regions, such as the heart base [
      • McWilliam A.
      • Kennedy J.
      • Hodgson C.
      • Vasquez E.
      • Faivre-Finn C.
      • Van H.M.
      Radiation dose to heart base linked with poorer survival in lung cancer patients.
      ,
      • McWilliam A.
      • Khalifa J.
      • Vasquez Osorio E.
      • Banfill K.
      • Abravan A.
      • Faivre-Finn C.
      • et al.
      Novel methodology to investigate the effect of radiation dose to heart substructures on overall survival.
      ,
      • McWilliam A.
      • Dootson C.
      • Graham L.
      • Banfill K.
      • Abravan A.
      • van Herk M.
      Dose surface maps of the heart can identify regions associated with worse survival for lung cancer patients treated with radiotherapy.
      ], may mediate this effect. Moreover, dose-volume characteristics for several of the specific cardiac substructures are important for predicting toxicity after conventional fractionation lung cancer RT [
      • Wang K.
      • Pearlstein K.A.
      • Patchett N.D.
      • Deal A.M.
      • Mavroidis P.
      • Jensen B.C.
      • et al.
      Heart dosimetric analysis of three types of cardiac toxicity in patients treated on dose-escalation trials for Stage III non-small-cell lung cancer.
      ,
      • Yegya-Raman N.
      • Wang K.
      • Kim S.
      • Reyhan M.
      • Deek M.P.
      • Sayan M.
      • et al.
      Dosimetric predictors of symptomatic cardiac events after conventional-dose chemoradiation therapy for inoperable NSCLC.
      ,
      • Atkins K.
      • Chaunzwa T.
      • Lamba N.
      • Bitterman D.
      • Rawal B.
      • Bredfeldt J.
      • et al.
      Association of left anterior descending coronary artery radiation dose with major adverse cardiac events and mortality in patients with non-small cell lung cancer.
      ,
      • Vivekanandan S.
      • Landau D.B.
      • Counsell N.
      • Warren D.R.
      • Khwanda A.
      • Rosen S.D.
      • et al.
      The impact of cardiac radiation dosimetry on survival after radiation therapy for non-small cell lung cancer.
      ,
      • Ma J.T.
      • Sun L.
      • Sun X.
      • Xiong Z.C.
      • Liu Y.
      • Zhang S.L.
      • et al.
      Is pulmonary artery a dose-limiting organ at risk in non-small cell lung cancer patients treated with definitive radiotherapy.
      ,
      • Han C.B.
      • Wang W.L.
      • Quint L.
      • Xue J.X.
      • Matuszak M.
      • Ten Haken R.
      • et al.
      Pulmonary artery invasion, high-dose radiation, and overall survival in patients with non-small cell lung cancer.
      ]. The cardiac substructures are challenging to contour on RT planning computed tomography (CT), due to geometry complexity, limited intracardiac soft tissue definition and cardiorespiratory motion artefact [
      • Morris E.D.
      • Ghanem A.I.
      • Zhu S.
      • Dong M.
      • Pantelic M.V.
      • Glide-Hurst C.K.
      Quantifying inter-fraction cardiac substructure displacement during radiotherapy via magnetic resonance imaging guidance.
      ]. The imminent requirement to incorporate substructures in treatment planning is an opportunity to embed a cardiac segmentation tool into RT workflows.
      To this end, in previous studies, a neural network was trained from 240 lung cancer treatment plans [
      • Haq R.
      • Hotca A.
      • Apte A.
      • Rimner A.
      • Deasy J.O.
      • Thor M.
      Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis.
      ] to generate a whole heart and pericardium structure, and 10 cardiac substructures, based on the most established cardiac atlas [
      • Feng M.
      • Moran J.M.
      • Koelling T.
      • Chughtai A.
      • Chan J.L.
      • Freedman L.
      • et al.
      Development and validation of a heart atlas to study cardiac exposure to radiation following treatment for breast cancer.
      ]. This tool delineates the structures on standard free-breathing three-dimensional planning CT scans (3D-CT) with a high performance in 10 s, regardless of whether there is contrast enhancement [
      • Haq R.
      • Hotca A.
      • Apte A.
      • Rimner A.
      • Deasy J.O.
      • Thor M.
      Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis.
      ]. The substructures delineated include the four cardiac chambers and four great vessels. When applied to the RTOG-0617 cohort, the whole heart algorithm from this tool was shown to represent the cardiac dosimetry more accurately than the original manual contours used in the trial [
      • Thor M.
      • Apte A.
      • Haq R.
      • Iyer A.
      • LoCastro E.
      • Deasy J.O.
      Using auto-segmentation to reduce contouring and dose inconsistency in clinical trials: the simulated impact on RTOG 0617.
      ].
      For improved motion management, the use of four-dimensional (4D) planning scans (4D-CT) has largely replaced 3D-CT planning in contemporary lung cancer RT [
      • Cole A.J.
      • Hanna G.G.
      • Jain S.
      • O’Sullivan J.M.
      Motion Management for Radical Radiotherapy in Non-small Cell Lung Cancer.
      ]. Validation of the deep learning auto-segmentation tool in the 4D setting is therefore required for its clinical implementation. In this study, we hypothesised that the performance of the tool in the average intensity projection of 4D-CT scans (4D-AVE) would be comparable to 3D-CT using geometry, volume, dosimetry and clinical acceptability metrics.

      2. Materials & methods

      2.1 Study design

      The 4D-AVE dataset for 20 patients that completed radical RT for lung cancer between 2015 and 2020 at a single centre were selected based on a random number generator for this retrospective study. RT treatment plans underwent manual and automated cardiac segmentation for comparison of the deep learning-based tool’s performance. This modest sample size was prospectively chosen given the resemblance of the research to a feasibility study, and due to the labour-intense nature of cardiac contouring. There was no stratification by presence/absence of contrast as the algorithm was initially trained using both scenarios (46 % contrast; 54 % non-contrast). Governance was granted and ethical approval was waived by the Belfast Health & Social Care Trust (IRAS ID 293181) and the study was sponsored by Queen’s University Belfast.

      2.2 Radiotherapy treatment plans

      Scans of 20 patients who underwent 4D-CT planning were included. Patients were scanned in the supine position, immobilised using a knee rest and thorax board, with arms holding a T-bar above their head. Scans were performed during quiet respiratory motion using the Varian RPM system (Varian Medical Systems, Palo Alto, CA, USA) with the GE Advantage Sim 4D application (GE Medical Systems, Milwaukee, WI, USA). Ten phase bins were created to generate the 4D-AVE used for this study. CT images at 2.5 mm slice width were acquired from the cricoid to the second lumbar vertebra with intravenous contrast when clinically appropriate. Volumetric modulated arc therapy (VMAT) treatment plans were calculated for all patients based on the phase-binned 4D-CT using the Varian AAA 13.6.23 algorithm on the Varian Eclipse treatment planning system.

      2.3 Manual segmentation

      Eight cardiac substructures were manually delineated as the gold standard for comparison, using the 4D-AVE for all patients using standard mediastinal window-level settings, or alternatives when necessary. The right atrium (RA), left atrium (LA), right ventricle (RV), left ventricle (LV), aorta (AO), pulmonary artery (PA), superior vena cava (SVC) and inferior vena cava (IVC) were contoured on Eclipse. A whole heart (WH) structure was also segmented, with the superior border defined as the most inferior CT slice showing the pulmonary trunk bifurcating into the left and right pulmonary arteries. A ‘pericardium’ (PC) structure was generated by duplicating the WH structure and extending this to the superior extent of the aortic arch. Composite bilateral atria and ventricle structures were generated using the union function. All manual delineations were completed by the same clinical oncologist (GW) using the Feng atlas [
      • Feng M.
      • Moran J.M.
      • Koelling T.
      • Chughtai A.
      • Chan J.L.
      • Freedman L.
      • et al.
      Development and validation of a heart atlas to study cardiac exposure to radiation following treatment for breast cancer.
      ] and were then verified, and modified if required, by a senior thoracic radiation oncologist (GH) and a senior cardiologist (CMC) simultaneously.

      2.4 Auto-Segmentation

      The 4D-AVE scans were imported to MATLAB version 2020b (The Mathworks, Inc., Massacusetts, USA), where the validated deep learning segmentation tool [
      • Haq R.
      • Hotca A.
      • Apte A.
      • Rimner A.
      • Deasy J.O.
      • Thor M.
      Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis.
      ] was applied. The deep neural network architecture was previously trained for label prediction of all pixels on individual CT slices as either within one of the cardiac structures, or the background image. Initially the anonymised CT data and structure (lungs only) files were uploaded within the Computational Environment for Radiotherapy Research (CERR) platform [
      • Deasy J.O.
      • Blanco A.I.
      • Clark V.H.
      CERR: a computational environment for radiotherapy research.
      ]. The CT image is cropped by the algorithm, using the medial walls and craniocaudal extent of the lung structures as limits on the region from which to begin generating structures. As part of the algorithm’s label prediction process, the input data undergoes random cropping, horizontal and vertical flipping, and rotation by 10°, as the segmentations are created. Once complete, the finalised individual slices are then stacked back together to generate a 3D segmentation. The time taken for the algorithm to complete is dependent on the computing hardware available, ranging from seconds only [
      • Haq R.
      • Hotca A.
      • Apte A.
      • Rimner A.
      • Deasy J.O.
      • Thor M.
      Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis.
      ] to approximately 10 minutes in our study.
      Automated structures were then imported back into Eclipse. Post-processing steps were applied to all structures, limited to removing structures < 0.5 cc (great vessels) or < 1.0 cc (all others) and the filling of cavities < 2 cc. Manual contours of the descending AO and IVC were cropped to the most inferior slice of the corresponding auto-contour, in order to pragmatically guarantee robust comparison of these structures, given that there are no published atlases for these substructures at the time of writing. Prior to geometric and dosimetric analysis, structures were amended on slices where a boundary was deemed to be suboptimal, as per section 2.6, with original versions retained for assessment of the impact of any changes.

      2.5 Quantitative evaluation

      Manual and automated structures were geometrically compared by percentage volume difference (VD), centroid shift (CS), Dice similarity coefficient (DSC), and 95 % Percentile Hausdorff distance (HD95) using Slicer-RT (PerkLab, Ontario, Canada) [
      • Pinter C.
      • Lasso A.
      • Wang A.
      • Jaffray D.
      • Fichtinger G.
      SlicerRT: radiation therapy research toolkit for 3D Slicer.
      ]. To evaluate dosimetric impact, the mean dose and maximum dose to 0.5 cc (Dmax) of the automated structures as calculated in Eclipse were compared against the manual contours.

      2.6 Qualitative evaluation

      A senior cardiologist (CMC) and senior thoracic radiation oncologist (GH) assessed the output of the auto-segmentation tool output structures following post-processing, prior to minor amendments, according to the scale used in the deep learning tool’s original publication [
      • Haq R.
      • Hotca A.
      • Apte A.
      • Rimner A.
      • Deasy J.O.
      • Thor M.
      Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis.
      ]. Based on their theoretical suitability for treatment planning including substructure dose constraints, individual structures were rated as ‘good’, ‘acceptable’, ‘in need of amendment’ (NOA) or ‘poor’, according to the number of slices requiring amendment (see Supplementary Table 1). Both ‘good’ and ‘acceptable’ ratings equate to a whole structure being of sufficient quality without further modification. The maximum number of slices that required modification to achieve a perfect delineation was also recorded.

      2.7 Statistical analysis

      Following data collection, statistics were calculated using Prism v8.3.0 (GraphPad Software, San Diego, California, USA). Median, maximum and minimum values are displayed and were used for significance testing as the vast majority of data were not normally distributed according to the Shapiro-Wilk test. The significance of differences between parameters in manual and automated contours were tested using the Mann-Whitney U test, used for comparing non-parametric, unpaired data. Bland-Altman analyses were undertaken to assess bias, and Spearman’s r values were calculated to assess correlation. For DSC and HD95 data, comparison was made with summary statistics from the original publication [
      • Haq R.
      • Hotca A.
      • Apte A.
      • Rimner A.
      • Deasy J.O.
      • Thor M.
      Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis.
      ] using Mann-Whitney tests, to allow benchmarking of the presented work.

      3. Results

      3.1 Patients

      The median age of the cohort was 70.5 years, 7 of whom were female. Additional patient characteristics are summarised in Supplementary Table 2. Intravenous contrast was administered for 70 %, and all patients exhibited calcification which was mild, moderate or severe in 25 %, 45 % and 30 % respectively. A 3D reconstruction of the automated and manual contours for a representative patient are included in Fig. 1.
      Figure thumbnail gr1
      Fig. 1Three-dimensional reconstruction of cardiac substructures from a representative patient from the anterior (A, B), posterior (C, D), left (E, F) and right (G, H) perspectives, based on manual (top) and automated delineations (bottom) (right atrium = cyan; left atrium = orange; right ventricle = blue; left ventricle = red; pulmonary artery = green; aorta = magenta; superior vena cava = yellow; inferior vena cava = brown)

      3.2 Comparison of geometry

      Automated volumes were smaller than manual for the chambers, except LV, and larger for the great vessels, except AO, as shown in Fig. 2A and Supplementary Table 3. The median VD for the WH was 1.3 % and the highest and lowest magnitude median VDs amongst the cardiac substructures were for the LV (6.1 %) and IVC (41.5 %) respectively. Statistically significant median VDs were found for RA, LA, RV and IVC. The overall median absolute VD across all substructures was 11.8 % (range 6.5–41.5). Comparing the volumes, there were low levels of bias as shown by good spread in the points, and few points out-with the limits of agreement for the majority of structures in Bland-Altman plots (see Supplementary Fig. 1). Values were also strongly correlated, with a mean Spearman’s r value of 0.90 across all cardiac structures (see Supplementary Table 4).
      Figure thumbnail gr2
      Fig. 2A) Box plot of percentage volume difference between the automated and manual delineations, relative to the manual contour. B) Box plot of DSCs for each substructure, comparing data from this manuscript with the original publication. (WH = whole heart; PC = pericardium; RA = right atrium; LA = left atrium; RV = right ventricle; LV = left ventricle; AO = aorta; PA = pulmonary artery; SVC = superior vena cava; IVC = inferior vena cava).
      As shown in Fig. 2B and Supplementary Table 5, DSC values ranged 0.76–0.94 across the structures, with the worst performance on average for IVC at 0.76, and the best for PC at 0.94. The median DSC across all structures was 0.85 and the median DSC across patients was 0.82, 0.92 and 0.86 for minimum, maximum and median of respectively. In addition, there was good similarity in the DSCs with the original published cohort (p = 0.27).
      Generally CS was approximately 3–4 mm for each structure, though was slightly better for AO at 1.7 mm and slightly worse for PA at 5.7 mm, as shown in Supplementary Table 6. Regarding the directionality of the X, Y and Z components of these shifts, there was a suggestion of a systematic effect in the X and Z axes, with negative values found for medians in 8/10 (ie right shifts) and 3/10 (ie inferior shifts) structures respectively. This was not observed in the Y axis (5/10 negative values) and moreover, as shown in Supplementary Table 7, the magnitude of the median shifts per structure were typically < 2 mm across substructures in all directions.
      The HD95 values for the automated structures on 4D-AVE were generally small, with a median of 7.1 mm, and the substructures with the lowest and highest HD95s being the AO (2.4 mm) and the RV (8.9 mm). These data were clinically comparable to the published 3D-CT data, which had a median and range of 5.9 mm and 3.1–7.3 mm. Further details on the HD95 values are available in Supplementary Table 8.

      3.3 Comparison of dose characteristics

      Mean and maximum doses to structures were not significantly different between the automated and manual segmentations on 4D-AVE scans, with a median absolute difference of 0.2 Gy (range 0–1.7 Gy) and 0.4 Gy (range 0.1–2.2 Gy) for mean dose and Dmax respectively. Differences were generally small for all cardiac structures, especially for mean dose, as shown in Supplementary Tables 9 and 10. Levels of correlation were high for both mean dose and Dmax, with median values of 0.99 and 0.96 respectively, as shown in Fig. 3. There were also low levels of bias, given the spread in the points in the Bland-Altman plots and low number of outlying points (see Supplementary Fig. 2, Fig. 3). The performance of the tool in each structure varied by parameter, but there was a pattern of lower dose differences for the RV and LV. Per patient across all structures, differences in mean dose were generally ± 2 Gy. Similarly for Dmax, differences were generally between −7 Gy and + 4 Gy across all structures per patient.
      Figure thumbnail gr3
      Fig. 3Scatter plots of mean (A) and maximum (B) doses (Gy) to automated and manual delineations, with Spearman correlation values displayed. (WH = whole heart; PC = pericardium; RA = right atrium; LA = left atrium; RV = right ventricle; LV = left ventricle; AO = aorta; PA = pulmonary artery; SVC = superior vena cava; IVC = inferior vena cava).

      3.4 Qualitative comparison

      Virtually all structures (99.5 %) and all patients (19 of 20) were deemed to be appropriate for clinical use without further editing by two senior clinicians (CMC, GH) according to criteria used by Haq et al, as shown in Fig. 4. Minor modifications of the automated contours (median of 2 slices for 18 % structures) led to insignificant changes in geometry only (see Supplementary Data File). Automated and manual delineations on the 4D-AVE scan are shown in Fig. 5.
      Figure thumbnail gr4
      Fig. 4Qualitative evaluation of automated segmentation of 4D-CT RT planning scans for the whole heart and individual cardiac substructures for 20 patients with lung cancer.
      Figure thumbnail gr5
      Fig. 5Representative cross-sectional images of the manual (purple) and automated (cyan) segmentations of the substructures on the average intensity projection scan, in the transverse (A–B), sagittal (C) and coronal (D) planes. (WH = whole heart; PC = pericardium; RA = right atrium; LA = left atrium; RV = right ventricle; LV = left ventricle; AO = aorta; PA = pulmonary artery; SVC = superior vena cava; IVC = inferior vena cava; NOA = in need of amendment). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

      4. Discussion

      Cardiac substructure auto-segmentation has come to the fore in recent years owing to an increased interest in radiation effects on the heart [
      • Banfill K.
      • Giuliani M.
      • Aznar M.
      • Franks K.
      • McWilliam A.
      • Schmitt M.
      • et al.
      Cardiac toxicity of thoracic radiotherapy: existing evidence and future directions.
      ,
      • Walls G.M.
      • Lyon A.R.
      • Harbinson M.T.
      • Hanna G.G.
      Cardiotoxicity following cancer treatment.
      ]. Recent studies have elicited dose–response relationships for several component structures [
      • Wang K.
      • Pearlstein K.A.
      • Patchett N.D.
      • Deal A.M.
      • Mavroidis P.
      • Jensen B.C.
      • et al.
      Heart dosimetric analysis of three types of cardiac toxicity in patients treated on dose-escalation trials for Stage III non-small-cell lung cancer.
      ,
      • Yegya-Raman N.
      • Wang K.
      • Kim S.
      • Reyhan M.
      • Deek M.P.
      • Sayan M.
      • et al.
      Dosimetric predictors of symptomatic cardiac events after conventional-dose chemoradiation therapy for inoperable NSCLC.
      ,
      • Atkins K.
      • Chaunzwa T.
      • Lamba N.
      • Bitterman D.
      • Rawal B.
      • Bredfeldt J.
      • et al.
      Association of left anterior descending coronary artery radiation dose with major adverse cardiac events and mortality in patients with non-small cell lung cancer.
      ,
      • Vivekanandan S.
      • Landau D.B.
      • Counsell N.
      • Warren D.R.
      • Khwanda A.
      • Rosen S.D.
      • et al.
      The impact of cardiac radiation dosimetry on survival after radiation therapy for non-small cell lung cancer.
      ,
      • Ma J.T.
      • Sun L.
      • Sun X.
      • Xiong Z.C.
      • Liu Y.
      • Zhang S.L.
      • et al.
      Is pulmonary artery a dose-limiting organ at risk in non-small cell lung cancer patients treated with definitive radiotherapy.
      ,
      • Han C.B.
      • Wang W.L.
      • Quint L.
      • Xue J.X.
      • Matuszak M.
      • Ten Haken R.
      • et al.
      Pulmonary artery invasion, high-dose radiation, and overall survival in patients with non-small cell lung cancer.
      ] and trials are underway to test the effect of sparing these regions [

      Faivre-Finn C, Franks K. ACCOLADE: avoiding cardiac toxicity in lung cancer patients treated with curative-intent radiotherapy (NCT03645317), 2018.

      ]. As the application of artificial intelligence is explored in RT, volume delineation serves as a logical starting point [
      • Wahid K.A.
      • Ahmed S.
      • He R.
      • van Dijk L.V.
      • Teuwen J.
      • McDonald B.A.
      • et al.
      Evaluation of deep learning-based multiparametric MRI oropharyngeal primary tumor auto-segmentation and investigation of input channel effects: Results from a prospective imaging registry.
      ,
      • Chen X.
      • Sun S.
      • Bai N.
      • Han K.
      • Liu Q.
      • Yao S.
      • et al.
      A deep learning-based auto-segmentation system for organs-at-risk on whole-body computed tomography images for radiation therapy.
      ,
      • Xie Y.
      • Kang K.
      • Wang Y.
      • Khandekar M.J.
      • Willers H.
      • Keane F.K.
      • et al.
      Automated clinical target volume delineation using deep 3d neural networks in radiation therapy of non-small cell lung cancer.
      ]. By accurately performing complex and time-consuming delineation tasks, clinician availability for alternate activities could be increased and treatment planning delays reduced [
      • Luk S.M.H.
      • Ford E.C.
      • Phillips M.H.
      • Kalet A.M.
      Improving the quality of care in radiation oncology using artificial intelligence.
      ].
      In this study, a deep learning-based cardiac substructure auto-segmentation tool developed for use in 3D-CT scans was retrospectively evaluated in 20 patients that underwent 4D-CT planning. This particular tool was selected from the literature [
      • Garrett Fernandes M.
      • Bussink J.
      • Stam B.
      • Wijsman R.
      • Schinagl D.A.X.
      • Monshouwer R.
      • et al.
      Deep learning model for automatic contouring of cardiovascular substructures on radiotherapy planning CT images: Dosimetric validation and reader study based clinical acceptability testing.
      ,
      • Kaderka R.
      • Gillespie E.F.
      • Mundt R.C.
      • Bryant A.K.
      • Sanudo-Thomas C.B.
      • Harrison A.L.
      • et al.
      Geometric and dosimetric evaluation of atlas based auto-segmentation of cardiac structures in breast cancer patients.
      ,
      • van Velzen S.G.M.
      • Bruns S.
      • Wolterink J.M.
      • Leiner T.
      • Viergever M.A.
      • Verkooijen H.M.
      • et al.
      AI-based quantification of planned radiation therapy dose to cardiac structures and coronary arteries in patients with breast cancer.
      ,
      • Farrugia M.
      • Yu H.
      • Singh A.K.
      • Malhotra H.
      Autosegmentation of cardiac substructures in respiratory-gated, non-contrasted computed tomography images.
      ,
      • Spoor D.S.
      • Sijtsema N.M.
      • van den Bogaard V.A.B.
      • van der Schaaf A.
      • Brouwer C.L.
      • Ta B.D.P.
      • et al.
      Validation of separate multi-atlases for auto segmentation of cardiac substructures in CT-scans acquired in deep inspiration breath hold and free breathing.
      ,
      • Maffei N.
      • Fiorini L.
      • Aluisio G.
      • D’Angelo E.
      • Ferrazza P.
      • Vanoni V.
      • et al.
      Hierarchical clustering applied to automatic atlas based segmentation of 25 cardiac sub-structures.
      ,
      • van den Bogaard V.A.B.
      • van Dijk L.V.
      • Vliegenthart R.
      • Sijtsema N.M.
      • Langendijk J.A.
      • Maduro J.H.
      • et al.
      Development and evaluation of an auto-segmentation tool for the left anterior descending coronary artery of breast cancer patients based on anatomical landmarks.
      ,
      • Loap P.
      • Servois V.
      • Dhonneur G.
      • Kirov K.
      • Fourquet A.
      • Kirova Y.
      A radiation therapy contouring atlas for cardiac conduction node delineation.
      ,
      • Loap P.
      • Tkatchenko N.
      • Kirova Y.
      Evaluation of a delineation software for cardiac atlas-based autosegmentation: an example of the use of artificial intelligence in modern radiotherapy.
      ,
      • Luo Y.
      • Xu Y.
      • Liao Z.
      • Gomez D.
      • Wang J.
      • Jiang W.
      • et al.
      Automatic segmentation of cardiac substructures from noncontrast CT images: accurate enough for dosimetric analysis?.
      ,
      • Morris E.D.
      • Ghanem A.I.
      • Pantelic M.V.
      • Walker E.M.
      • Han X.
      • Glide-Hurst C.K.
      Cardiac substructure segmentation and dosimetry using a novel hybrid magnetic resonance and computed tomography cardiac atlas.
      ,
      • Finnegan R.
      • Dowling J.
      • Koh E.S.
      • Tang S.
      • Otton J.
      • Delaney G.
      • et al.
      Feasibility of multi-atlas cardiac segmentation from thoracic planning CT in a probabilistic framework.
      ] due to its superiority in terms of structures included, performance metrics, applicability to lung cancer and availability. The cardiac substructure auto-segmentation available tools at the time of writing are compared in Supplementary Table 11. The median DSC across all substructures for the presented tool evaluated on 4D-CT scans compares well with other approaches, as shown in the heatmap below (Fig. 6). Of note, some papers from Supplementary Table 11 are not represented as none of the presented tool’s substructures were included, or because DSC were not reported. Furthermore, the selected articles were heterogenous in their calculation of summary statistics from individual cases, with some using mean and others using median.
      Figure thumbnail gr6
      Fig. 6A heat map of DSCs among studies evaluating novel auto-segmentation tools for the cardiac substructures. (WH = whole heart; RA = right atrium; LA = left atrium; RV = right ventricle; LV = left ventricle; AO = aorta; PA = pulmonary artery; SVC = superior vena cava; IVC = inferior vena cava).
      In evaluating the geometric, dosimetric and clinical acceptability metrics of the deep learning tool on the 4D-AVE scans, the automated contours were found to be comparable with manual contours, with median DSC, centroid shift and mean dose difference of 0.85, 3.6 mm and 0.2 Gy. The overall performance of the tool per substructure varied according to parameter observed, for example there was a trend for lower performance on the RV (low DSC, high VD, high HD95) and higher performance on the PA (high DSC, small VD, low HD95).
      Several of the structures with lower similarity between manual and auto-segmentation are challenging to delineate reproducibly as there is no atlas (e.g. for PA) or have an uncertain anatomical boundary on CT (e.g. for RA and LA), explaining the lower performance. The narrow inter-patient range in all of the parameter’s averages considering all substructures suggests that patients are contoured with similar degrees of ‘ease’ or ‘difficulty’. However, some structures had consistent differences in volume or dose across most patients (e.g. the RA had higher volumes and the SVC had lower mean doses). Interestingly, those structures with larger VDs did not appear to have larger dose differences.
      Reassuringly overall, differences in the summary statistics between our 4D-CT data and the original 3D-CT publication were clinically insignificant, despite manual segmentations being completed by different operators [
      • Haq R.
      • Hotca A.
      • Apte A.
      • Rimner A.
      • Deasy J.O.
      • Thor M.
      Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis.
      ]. Of note, the automated contours were found to be clinically acceptable overall, including those structures with larger VDs. Furthermore, clinician adjustment appeared to be unwarranted as geometric impact of resulting adjustments were shown to be negligible (see Supplementary Data File). The only anomaly with any frequency for the neural network was delineating the inferior WH and PC slices, where there is poor soft tissue contrast and considerable motion artefact at the interface with the diaphragm.
      It is noteworthy that the presented auto-segmentation tool overcame the challenges of cardiac substructure delineation to produce contours equivalent to manual contours. The impediments include complex anatomy, frequent absence of contrast enhancement, poor soft tissue definition eg between the myocardium and pericardial fat, and biological variability eg vascular calcification or aneurysm. In addition, by using the 4D-AVE scan, motion artefacts caused by substructure deformation secondary to lung movement and heart contractions are present as aforementioned.
      Although modern RT planning accounts for lung cancer movement associated with breathing, cardiac motion is not routinely accounted for during these processes. In a study using the maximum intensity projection (MIP) from 4D-CT, planning risk volume (PRV) margins of 5.8 mm and 4.8 mm were recommended for compensation of whole heart motion in the lateral and cranio-caudal axes [
      • Vasquez Osorio E.M.
      • McCallum H.
      • Bedair A.
      • Faivre-Finn C.
      • Haughey A.
      • van Herk M.
      • et al.
      Protecting the heart: a practical approach to account for the full extent of heart motion in radiation therapy planning.
      ]. However, independent displacement of the individual cardiac substructures is likely to be under-estimated by whole heart margins. Virtually any plane through the heart contains several substructures exhibiting non-synchronised and non-isotropic deformation patterns. Due to reciprocal compensation of motion, and buffering of exterior displacement by the surrounding pericardial lining, total displacement of substructures is therefore likely to be larger than the total heart margin. Furthermore, substructure motion will not necessarily be oscillatory after interactions from concurrent respiratory motion are considered. With gold standard cardiac-gated CT in breath-hold, where lung motion was obviated, centroid shifts for select substructures were limited to 0.5–1.6 mm [
      • Ouyang Z.
      • Schoenhagen P.
      • Wazni O.
      • Tchou P.
      • Saliba W.I.
      • Suh J.H.
      • et al.
      Analysis of cardiac motion without respiratory motion for cardiac stereotactic body radiation therapy.
      ].
      Beyond cardiac motion, other unresolved issues in cardiac substructure segmentation include the lack of standardised great vessel definitions, as these are not defined in the published cardiac atlases [
      • Feng M.
      • Moran J.M.
      • Koelling T.
      • Chughtai A.
      • Chan J.L.
      • Freedman L.
      • et al.
      Development and validation of a heart atlas to study cardiac exposure to radiation following treatment for breast cancer.
      ,
      • Loap P.
      • Servois V.
      • Dhonneur G.
      • Kirov K.
      • Fourquet A.
      • Kirova Y.
      A radiation therapy contouring atlas for cardiac conduction node delineation.
      ,
      • Duane F.
      • Aznar M.C.
      • Bartlett F.
      • Cutter D.J.
      • Darby S.C.
      • Jagsi R.
      • et al.
      A cardiac contouring atlas for radiotherapy.
      ,
      • Milo M.L.H.
      • Offersen B.V.
      • Bechmann T.
      • Diederichsen A.C.P.
      • Hansen C.R.
      • Holtved E.
      • et al.
      Delineation of whole heart and substructures in thoracic radiation therapy: national guidelines and contouring atlas by the Danish Multidisciplinary Cancer Groups.
      ,
      • Socha J.
      • Rygielska A.
      • Uziębło-Życzkowska B.
      • Chałubińska-Fendler J.
      • Jurek A.
      • Maciorowska M.
      • et al.
      A heart valves contouring atlas on average intensity projection 4D-CT for lung cancer radiotherapy.
      ]. Current atlases also do not recommend subtraction of the cardiac chamber blood pool, which is likely to be confounding how the dose metrics of the cardiac chamber muscles are interpreted [
      • van Velzen S.G.M.
      • Bruns S.
      • Wolterink J.M.
      • Leiner T.
      • Viergever M.A.
      • Verkooijen H.M.
      • et al.
      AI-based quantification of planned radiation therapy dose to cardiac structures and coronary arteries in patients with breast cancer.
      ,
      • Finnegan R.
      • Dowling J.
      • Koh E.S.
      • Tang S.
      • Otton J.
      • Delaney G.
      • et al.
      Feasibility of multi-atlas cardiac segmentation from thoracic planning CT in a probabilistic framework.
      ]. It will be crucial to have readily reproducible substructure definitions, given the pervasively difficult but clinically crucial task of reducing interobserver variation [
      • Tsang Y.
      • Hoskin P.
      • Spezi E.
      • Landau D.
      • Lester J.
      • Miles E.
      • et al.
      Assessment of contour variability in target volumes and organs at risk in lung cancer radiotherapy.
      ]. Similarly, there is a lack of guidance on how substructure studies should handle collinearity in the analysis of dose-volume statistics of intersecting structures such as the conduction system and atria, or the distal coronary arteries and myocardium.
      Two alternate auto-segmentation algorithms that use a neural network architecture have been published, subsequent to the development of the tool presented, trained using 3D-CT planning cases. The model published by Garrett Fernandes et al had DSC of 0.74–0.95 across the same set of substructures and the largest median absolute difference in mean doses in the range 0.1 Gy–1.0 Gy [
      • Garrett Fernandes M.
      • Bussink J.
      • Stam B.
      • Wijsman R.
      • Schinagl D.A.X.
      • Monshouwer R.
      • et al.
      Deep learning model for automatic contouring of cardiovascular substructures on radiotherapy planning CT images: Dosimetric validation and reader study based clinical acceptability testing.
      ]. The model published by Van Velzen et al had a DSC of 0.76–0.88 and R2 values for dosimetric parameters were 0.77–1.00 [
      • van Velzen S.G.M.
      • Lessmann N.
      • Velthuis B.K.
      • Bank I.E.M.
      • van den Bongard D.H.J.G.
      • Leiner T.
      • et al.
      Deep learning for automatic calcium scoring in CT: Validation using multiple cardiac CT and chest CT protocols.
      ]. These values mirror those in the original description of the presented tool and in this updated 4D-CT test cohort, with an overall DSC of 0.76–0.93 and mean dose differences and R2 values of −1.6–0.3 Gy and 0.96–1.00. Taken together, the deep learning-based tool utilised in this study is suitable for use in 4D-CT RT planning scans, even though it was trained on a 3D-CT dataset.
      The main strength of this study is that it is the first to apply what is the most widely applied substructure tool at the time of writing, a tool which is based on the most established cardiac atlas, and is widely applicable owing to its ‘open source’ availability, to 4D-CT. We are the second group to demonstrate the feasibility of cardiac substructure auto-segmentation on 4D-CT scans, and our results corroborate the findings of those investigators in that 4D-CT is a suitable planning scan modality for cardiac substructure auto-contouring [
      • Luo Y.
      • Xu Y.
      • Liao Z.
      • Gomez D.
      • Wang J.
      • Jiang W.
      • et al.
      Automatic segmentation of cardiac substructures from noncontrast CT images: accurate enough for dosimetric analysis?.
      ,
      • Zhou R.
      • Liao Z.
      • Pan T.
      • Milgrom S.A.
      • Pinnix C.C.
      • Shi A.
      • et al.
      Cardiac atlas development and validation for automatic segmentation of cardiac substructures.
      ]. In those studies the mean DSC for the same structures was 0.88 [
      • Luo Y.
      • Xu Y.
      • Liao Z.
      • Gomez D.
      • Wang J.
      • Jiang W.
      • et al.
      Automatic segmentation of cardiac substructures from noncontrast CT images: accurate enough for dosimetric analysis?.
      ] and 0.79 [
      • Zhou R.
      • Liao Z.
      • Pan T.
      • Milgrom S.A.
      • Pinnix C.C.
      • Shi A.
      • et al.
      Cardiac atlas development and validation for automatic segmentation of cardiac substructures.
      ] across substructures, compared with our 0.85. Furthermore, this 4D-CT cohort included a blend of contrast-enhanced and non-contrast scans of patients with varying degrees of cardiac calcification.
      The main limitations of this study are the cohort size and restriction to lung cancer cases. The 4D-AVE was used rather than the MIP despite prior work demonstrating the whole heart on 4D-AVE requires larger PRV margins than on MIP [
      • Vasquez Osorio E.M.
      • McCallum H.
      • Bedair A.
      • Faivre-Finn C.
      • Haughey A.
      • van Herk M.
      • et al.
      Protecting the heart: a practical approach to account for the full extent of heart motion in radiation therapy planning.
      ]. However, 4D-AVE was chosen for its superior soft tissue definition suited to delineating the complex geometries of the cardiac substructures, and for its similarity to the 3D-CT scan, on which our deep learning tool was based. DSC values for IVC and PA were relatively low compared to the other cardiac structures, but the dosimetric impact of this was low in this study, as has been found previously [
      • Vaassen F.
      • Hazelaar C.
      • Canters R.
      • Peeters S.
      • Petit S.
      • van Elmpt W.
      The impact of organ-at-risk contour variations on automatically generated treatment plans for NSCLC.
      ]. Although PRV margins have been proposed for the coronary arteries [
      • Levis M.
      • Filippi A.R.
      • Fiandra C.
      • De Luca V.
      • Bartoncini S.
      • Vella D.
      • et al.
      Inclusion of heart substructures in the optimization process of volumetric modulated arc therapy techniques may reduce the risk of heart disease in Hodgkin’s lymphoma patients.
      ], none have been published for the cardiac substructures examined in this study and so these were not added. Finally, the auto-contouring tool has several minor weaknesses, largely related to ambiguity in the atlas on which the tool is based, as listed in Supplementary Table 10.
      The development of auto-contouring tools for the cardiac substructures is warranted, because although atlases can reduce inter-observer variability (with or without contrast equally), the time required is significant. Moreover, rapid deep learning analysis made possible by robust tools such as the one presented could accelerate ‘big data’ analyses [
      • Kaderka R.
      • Gillespie E.F.
      • Mundt R.C.
      • Bryant A.K.
      • Sanudo-Thomas C.B.
      • Harrison A.L.
      • et al.
      Geometric and dosimetric evaluation of atlas based auto-segmentation of cardiac structures in breast cancer patients.
      ] such as interrogation of real-world data in substructure dose constraint investigations. Refinement of the historical clinical radiation response of the heart is required since the Gagliardi model involved limited endpoints and did not include patients with lung cancer [
      • Gagliardi G.
      • Constine L.S.
      • Moiseenko V.
      • Correa C.
      • Pierce L.J.
      • Allen A.M.
      • et al.
      Radiation dose-volume effects in the heart.
      ]. These efforts should be multidisciplinary, spanning all thoracic tumor types, with aligned clinical endpoints and adjustment for relevant comorbidities.

      5. Funding statement

      This work was performed within the Irish Clinical Academic Training (ICAT) Programme, supported by the Wellcome Trust and the Health Research Board (Grant No 203930/B/16/Z), the Health Service Executive National Doctors Training and Planning and the Health and Social Care, Research and Development Division, Northern Ireland.

      Declaration of Competing Interest

      The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

      Appendix A. Supplementary data

      The following are the Supplementary data to this article:

      References

        • Teoh S.
        • Fiorini F.
        • George B.
        • Vallis K.A.
        • Van den Heuvel F.
        Proton vs photon: a model-based approach to patient selection for reduction of cardiac toxicity in locally advanced lung cancer.
        Radiother Oncol. 2020; 152: 151-162https://doi.org/10.1016/j.radonc.2019.06.032
        • Thor M.
        • Deasy J.
        • Iyer A.
        • Bendau E.
        • Fontanella A.
        • Apte A.
        • et al.
        Toward personalized dose-prescription in locally advanced non-small cell lung cancer: validation of published normal tissue complication probability models.
        Radiother Oncol. 2019; 138: 45-51https://doi.org/10.1016/j.radonc.2019.05.011
        • Thind K.
        • Roumeliotis M.
        • Mann T.
        • Van Dyke L.
        • Martell K.
        • Smith W.
        • et al.
        Increasing demand on human capital and resource utilization in radiation therapy: the past decade.
        Int J Radiat Oncol. 2022; 112: 457-462https://doi.org/10.1016/j.ijrobp.2021.09.020
        • Vinod S.K.
        • Jameson M.G.
        • Min M.
        • Holloway L.C.
        Uncertainties in volume delineation in radiation oncology: a systematic review and recommendations for future studies.
        Radiother Oncol. 2016; 121: 169-179https://doi.org/10.1016/j.radonc.2016.09.009
        • Jameson M.G.
        • Kumar S.
        • Vinod S.K.
        • Metcalfe P.E.
        • Holloway L.C.
        Correlation of contouring variation with modeled outcome for conformal non-small cell lung cancer radiotherapy.
        Radiother Oncol. 2014; 112: 332-336https://doi.org/10.1016/j.radonc.2014.03.019
        • Kumar V.
        • Gu Y.
        • Basu S.
        • Berglund A.
        • Eschrich S.A.
        • Schabath M.B.
        • et al.
        QIN “radiomics: the process and the challenges”.
        Magn Reson Imaging. 2013; 30: 1234-1248https://doi.org/10.1016/j.mri.2012.06.010.QIN
        • Brouwer C.L.
        • Boukerroui D.
        • Oliveira J.
        • Looney P.
        • Steenbakkers R.J.H.M.
        • Langendijk J.A.
        • et al.
        Assessment of manual adjustment performed in clinical practice following deep learning contouring for head and neck organs at risk in radiotherapy.
        Phys Imaging Radiat Oncol. 2020; 16: 54-60https://doi.org/10.1016/j.phro.2020.10.001
        • Mohammadi R.
        • Shokatian I.
        • Salehi M.
        • Arabi H.
        • Shiri I.
        • Zaidi H.
        Deep learning-based auto-segmentation of organs at risk in high-dose rate brachytherapy of cervical cancer.
        Radiother Oncol. 2021; 159: 231-240https://doi.org/10.1016/j.radonc.2021.03.030
        • Atkins K.M.
        • Rawal B.
        • Chaunzwa T.L.
        • Lamba N.
        • Bitterman D.S.
        • Williams C.L.
        • et al.
        Cardiac radiation dose, cardiac disease, and mortality in patients with lung cancer.
        J Am Coll Cardiol. 2019; 73: 2976-2987https://doi.org/10.1016/j.jacc.2019.03.500
        • Speirs C.K.
        • DeWees T.A.
        • Rehman S.
        • Molotievschi A.
        • Velez M.A.
        • Mullen D.
        • et al.
        Heart dose is an independent dosimetric predictor of overall survival in locally advanced non-small cell lung cancer.
        J Thorac Oncol. 2017; 12: 293-301https://doi.org/10.1016/j.jtho.2016.09.134
        • Vivekanandan S.
        • Fenwick J.D.
        • Counsell N.
        • Panakis N.
        • Stuart R.
        • Higgins G.S.
        • et al.
        Associations between cardiac irradiation and survival in patients with non-small cell lung cancer: validation and new discoveries in an independent dataset.
        Radiother Oncol. 2021; 165: 119-125https://doi.org/10.1016/j.radonc.2021.10.016
        • McWilliam A.
        • Kennedy J.
        • Hodgson C.
        • Vasquez E.
        • Faivre-Finn C.
        • Van H.M.
        Radiation dose to heart base linked with poorer survival in lung cancer patients.
        Eur J Cancer. 2017; 85: 106-113https://doi.org/10.1016/j.ejca.2017.07.053
        • McWilliam A.
        • Khalifa J.
        • Vasquez Osorio E.
        • Banfill K.
        • Abravan A.
        • Faivre-Finn C.
        • et al.
        Novel methodology to investigate the effect of radiation dose to heart substructures on overall survival.
        Int J Radiat Oncol Biol Phys. 2020; https://doi.org/10.1016/j.ijrobp.2020.06.031
        • McWilliam A.
        • Dootson C.
        • Graham L.
        • Banfill K.
        • Abravan A.
        • van Herk M.
        Dose surface maps of the heart can identify regions associated with worse survival for lung cancer patients treated with radiotherapy.
        Phys Imaging Radiat Oncol. 2020; 15: 46-51https://doi.org/10.1016/j.phro.2020.07.002
        • Wang K.
        • Pearlstein K.A.
        • Patchett N.D.
        • Deal A.M.
        • Mavroidis P.
        • Jensen B.C.
        • et al.
        Heart dosimetric analysis of three types of cardiac toxicity in patients treated on dose-escalation trials for Stage III non-small-cell lung cancer.
        Radiother Oncol. 2017; 125: 293-300https://doi.org/10.1016/j.radonc.2017.10.001
        • Yegya-Raman N.
        • Wang K.
        • Kim S.
        • Reyhan M.
        • Deek M.P.
        • Sayan M.
        • et al.
        Dosimetric predictors of symptomatic cardiac events after conventional-dose chemoradiation therapy for inoperable NSCLC.
        J Thorac Oncol. 2018; 13: 1508-1518https://doi.org/10.1016/j.jtho.2018.05.028
        • Atkins K.
        • Chaunzwa T.
        • Lamba N.
        • Bitterman D.
        • Rawal B.
        • Bredfeldt J.
        • et al.
        Association of left anterior descending coronary artery radiation dose with major adverse cardiac events and mortality in patients with non-small cell lung cancer.
        JAMA Oncol. 2020; 02115: 1-13https://doi.org/10.1001/jamaoncol.2020.6332
        • Vivekanandan S.
        • Landau D.B.
        • Counsell N.
        • Warren D.R.
        • Khwanda A.
        • Rosen S.D.
        • et al.
        The impact of cardiac radiation dosimetry on survival after radiation therapy for non-small cell lung cancer.
        Int J Radiat Oncol Biol Phys. 2017; 99: 51-60https://doi.org/10.1016/j.ijrobp.2017.04.026
        • Ma J.T.
        • Sun L.
        • Sun X.
        • Xiong Z.C.
        • Liu Y.
        • Zhang S.L.
        • et al.
        Is pulmonary artery a dose-limiting organ at risk in non-small cell lung cancer patients treated with definitive radiotherapy.
        Radiat Oncol. 2017; 12: 4-13https://doi.org/10.1186/s13014-017-0772-5
        • Han C.B.
        • Wang W.L.
        • Quint L.
        • Xue J.X.
        • Matuszak M.
        • Ten Haken R.
        • et al.
        Pulmonary artery invasion, high-dose radiation, and overall survival in patients with non-small cell lung cancer.
        Int J Radiat Oncol Biol Phys. 2014; 89: 313-321https://doi.org/10.1016/j.ijrobp.2014.02.011
        • Morris E.D.
        • Ghanem A.I.
        • Zhu S.
        • Dong M.
        • Pantelic M.V.
        • Glide-Hurst C.K.
        Quantifying inter-fraction cardiac substructure displacement during radiotherapy via magnetic resonance imaging guidance.
        Phys Imaging Radiat Oncol. 2021; 18: 34-40https://doi.org/10.1016/j.phro.2021.03.005
        • Haq R.
        • Hotca A.
        • Apte A.
        • Rimner A.
        • Deasy J.O.
        • Thor M.
        Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis.
        Phys Imaging Radiat Oncol. 2020; 14: 61-66https://doi.org/10.1016/j.phro.2020.05.009
        • Feng M.
        • Moran J.M.
        • Koelling T.
        • Chughtai A.
        • Chan J.L.
        • Freedman L.
        • et al.
        Development and validation of a heart atlas to study cardiac exposure to radiation following treatment for breast cancer.
        Int J Radiat Oncol Biol Phys. 2011; 79: 10-18https://doi.org/10.1016/j.ijrobp.2009.10.058
        • Thor M.
        • Apte A.
        • Haq R.
        • Iyer A.
        • LoCastro E.
        • Deasy J.O.
        Using auto-segmentation to reduce contouring and dose inconsistency in clinical trials: the simulated impact on RTOG 0617.
        Int J Radiat Oncol Biol Phys. 2021; 109: 1619-1626https://doi.org/10.1016/j.ijrobp.2020.11.011
        • Cole A.J.
        • Hanna G.G.
        • Jain S.
        • O’Sullivan J.M.
        Motion Management for Radical Radiotherapy in Non-small Cell Lung Cancer.
        Clin Oncol (R Coll Radiol). 2014; 26: 67-80https://doi.org/10.1016/j.clon.2013.11.001
        • Deasy J.O.
        • Blanco A.I.
        • Clark V.H.
        CERR: a computational environment for radiotherapy research.
        Med Phys. 2003; 30: 979-985https://doi.org/10.1118/1.1568978
        • Pinter C.
        • Lasso A.
        • Wang A.
        • Jaffray D.
        • Fichtinger G.
        SlicerRT: radiation therapy research toolkit for 3D Slicer.
        Med Phys. 2012; 39: 6332-6338https://doi.org/10.1118/1.4754659
        • Banfill K.
        • Giuliani M.
        • Aznar M.
        • Franks K.
        • McWilliam A.
        • Schmitt M.
        • et al.
        Cardiac toxicity of thoracic radiotherapy: existing evidence and future directions.
        J Thorac Oncol. 2021; 16: 216-227https://doi.org/10.1016/j.jtho.2020.11.002
        • Walls G.M.
        • Lyon A.R.
        • Harbinson M.T.
        • Hanna G.G.
        Cardiotoxicity following cancer treatment.
        Ulster Med J. 2017; 86
      1. Faivre-Finn C, Franks K. ACCOLADE: avoiding cardiac toxicity in lung cancer patients treated with curative-intent radiotherapy (NCT03645317), 2018.

        • Wahid K.A.
        • Ahmed S.
        • He R.
        • van Dijk L.V.
        • Teuwen J.
        • McDonald B.A.
        • et al.
        Evaluation of deep learning-based multiparametric MRI oropharyngeal primary tumor auto-segmentation and investigation of input channel effects: Results from a prospective imaging registry.
        Clin Transl Radiat Oncol. 2022; 32: 6-14https://doi.org/10.1016/j.ctro.2021.10.003
        • Chen X.
        • Sun S.
        • Bai N.
        • Han K.
        • Liu Q.
        • Yao S.
        • et al.
        A deep learning-based auto-segmentation system for organs-at-risk on whole-body computed tomography images for radiation therapy.
        Radiother Oncol. 2021; 160: 175-184https://doi.org/10.1016/j.radonc.2021.04.019
        • Xie Y.
        • Kang K.
        • Wang Y.
        • Khandekar M.J.
        • Willers H.
        • Keane F.K.
        • et al.
        Automated clinical target volume delineation using deep 3d neural networks in radiation therapy of non-small cell lung cancer.
        Phys Imaging Radiat Oncol. 2021; 19: 131-137https://doi.org/10.1016/j.phro.2021.08.003
        • Luk S.M.H.
        • Ford E.C.
        • Phillips M.H.
        • Kalet A.M.
        Improving the quality of care in radiation oncology using artificial intelligence.
        Clin Oncol. 2021; https://doi.org/10.1016/j.clon.2021.11.011
        • Garrett Fernandes M.
        • Bussink J.
        • Stam B.
        • Wijsman R.
        • Schinagl D.A.X.
        • Monshouwer R.
        • et al.
        Deep learning model for automatic contouring of cardiovascular substructures on radiotherapy planning CT images: Dosimetric validation and reader study based clinical acceptability testing.
        Radiother Oncol. 2021; 165: 52-59https://doi.org/10.1016/j.radonc.2021.10.008
        • Kaderka R.
        • Gillespie E.F.
        • Mundt R.C.
        • Bryant A.K.
        • Sanudo-Thomas C.B.
        • Harrison A.L.
        • et al.
        Geometric and dosimetric evaluation of atlas based auto-segmentation of cardiac structures in breast cancer patients.
        Radiother Oncol. 2019; 131: 215-220https://doi.org/10.1016/j.radonc.2018.07.013
        • van Velzen S.G.M.
        • Bruns S.
        • Wolterink J.M.
        • Leiner T.
        • Viergever M.A.
        • Verkooijen H.M.
        • et al.
        AI-based quantification of planned radiation therapy dose to cardiac structures and coronary arteries in patients with breast cancer.
        Int J Radiat Oncol Biol Phys. 2022; 112: 611-620https://doi.org/10.1016/j.ijrobp.2021.09.009
        • Farrugia M.
        • Yu H.
        • Singh A.K.
        • Malhotra H.
        Autosegmentation of cardiac substructures in respiratory-gated, non-contrasted computed tomography images.
        World J Clin Oncol. 2021; 12: 95-102https://doi.org/10.5306/wjco.v12.i2.95
        • Spoor D.S.
        • Sijtsema N.M.
        • van den Bogaard V.A.B.
        • van der Schaaf A.
        • Brouwer C.L.
        • Ta B.D.P.
        • et al.
        Validation of separate multi-atlases for auto segmentation of cardiac substructures in CT-scans acquired in deep inspiration breath hold and free breathing.
        Radiother Oncol. 2021; 163: 46-54https://doi.org/10.1016/j.radonc.2021.07.025
        • Maffei N.
        • Fiorini L.
        • Aluisio G.
        • D’Angelo E.
        • Ferrazza P.
        • Vanoni V.
        • et al.
        Hierarchical clustering applied to automatic atlas based segmentation of 25 cardiac sub-structures.
        Phys Med. 2020; 69: 70-80https://doi.org/10.1016/j.ejmp.2019.12.001
        • van den Bogaard V.A.B.
        • van Dijk L.V.
        • Vliegenthart R.
        • Sijtsema N.M.
        • Langendijk J.A.
        • Maduro J.H.
        • et al.
        Development and evaluation of an auto-segmentation tool for the left anterior descending coronary artery of breast cancer patients based on anatomical landmarks.
        Radiother Oncol. 2019; 136: 15-20https://doi.org/10.1016/j.radonc.2019.03.013
        • Loap P.
        • Servois V.
        • Dhonneur G.
        • Kirov K.
        • Fourquet A.
        • Kirova Y.
        A radiation therapy contouring atlas for cardiac conduction node delineation.
        Pract Radiat Oncol. 2021; 11 (e434–7)https://doi.org/10.1016/j.prro.2021.02.002
        • Loap P.
        • Tkatchenko N.
        • Kirova Y.
        Evaluation of a delineation software for cardiac atlas-based autosegmentation: an example of the use of artificial intelligence in modern radiotherapy.
        Cancer/Radiotherapie. 2020; 24: 826-833https://doi.org/10.1016/j.canrad.2020.04.012
        • Luo Y.
        • Xu Y.
        • Liao Z.
        • Gomez D.
        • Wang J.
        • Jiang W.
        • et al.
        Automatic segmentation of cardiac substructures from noncontrast CT images: accurate enough for dosimetric analysis?.
        Acta Oncol (Madr). 2019; 58: 81-87https://doi.org/10.1080/0284186X.2018.1521985
        • Morris E.D.
        • Ghanem A.I.
        • Pantelic M.V.
        • Walker E.M.
        • Han X.
        • Glide-Hurst C.K.
        Cardiac substructure segmentation and dosimetry using a novel hybrid magnetic resonance and computed tomography cardiac atlas.
        Int J Radiat Oncol Biol Phys. 2019; 103: 985-993https://doi.org/10.1016/j.ijrobp.2018.11.025
        • Finnegan R.
        • Dowling J.
        • Koh E.S.
        • Tang S.
        • Otton J.
        • Delaney G.
        • et al.
        Feasibility of multi-atlas cardiac segmentation from thoracic planning CT in a probabilistic framework.
        Phys Med Biol. 2019; 64https://doi.org/10.1088/1361-6560/ab0ea6
        • Vasquez Osorio E.M.
        • McCallum H.
        • Bedair A.
        • Faivre-Finn C.
        • Haughey A.
        • van Herk M.
        • et al.
        Protecting the heart: a practical approach to account for the full extent of heart motion in radiation therapy planning.
        Int J Radiat Oncol Biol Phys. 2020; 108: 1082-1090https://doi.org/10.1016/j.ijrobp.2020.06.068
        • Ouyang Z.
        • Schoenhagen P.
        • Wazni O.
        • Tchou P.
        • Saliba W.I.
        • Suh J.H.
        • et al.
        Analysis of cardiac motion without respiratory motion for cardiac stereotactic body radiation therapy.
        J Appl Clin Med Phys. 2020; 21: 48-55https://doi.org/10.1002/acm2.13002
        • Duane F.
        • Aznar M.C.
        • Bartlett F.
        • Cutter D.J.
        • Darby S.C.
        • Jagsi R.
        • et al.
        A cardiac contouring atlas for radiotherapy.
        Radiother Oncol. 2017; 122: 416-422https://doi.org/10.1016/j.radonc.2017.01.008
        • Milo M.L.H.
        • Offersen B.V.
        • Bechmann T.
        • Diederichsen A.C.P.
        • Hansen C.R.
        • Holtved E.
        • et al.
        Delineation of whole heart and substructures in thoracic radiation therapy: national guidelines and contouring atlas by the Danish Multidisciplinary Cancer Groups.
        Radiother Oncol. 2020; 150: 121-127https://doi.org/10.1016/j.radonc.2020.06.015
        • Socha J.
        • Rygielska A.
        • Uziębło-Życzkowska B.
        • Chałubińska-Fendler J.
        • Jurek A.
        • Maciorowska M.
        • et al.
        A heart valves contouring atlas on average intensity projection 4D-CT for lung cancer radiotherapy.
        Radiother Oncol. 2021; 161 (S975–S7)https://doi.org/10.1016/s0167-8140(21)07628-3
        • Tsang Y.
        • Hoskin P.
        • Spezi E.
        • Landau D.
        • Lester J.
        • Miles E.
        • et al.
        Assessment of contour variability in target volumes and organs at risk in lung cancer radiotherapy.
        Tech Innov Patient Support Radiat Oncol. 2019; 10: 8-12https://doi.org/10.1016/j.tipsro.2019.05.001
        • van Velzen S.G.M.
        • Lessmann N.
        • Velthuis B.K.
        • Bank I.E.M.
        • van den Bongard D.H.J.G.
        • Leiner T.
        • et al.
        Deep learning for automatic calcium scoring in CT: Validation using multiple cardiac CT and chest CT protocols.
        Radiology. 2020; 295: 66-79https://doi.org/10.1148/radiol.2020191621
        • Zhou R.
        • Liao Z.
        • Pan T.
        • Milgrom S.A.
        • Pinnix C.C.
        • Shi A.
        • et al.
        Cardiac atlas development and validation for automatic segmentation of cardiac substructures.
        Radiother Oncol. 2017; 122: 66-71https://doi.org/10.1016/j.radonc.2016.11.016
        • Vaassen F.
        • Hazelaar C.
        • Canters R.
        • Peeters S.
        • Petit S.
        • van Elmpt W.
        The impact of organ-at-risk contour variations on automatically generated treatment plans for NSCLC.
        Radiother Oncol. 2021; 163: 136-142https://doi.org/10.1016/j.radonc.2021.08.014
        • Levis M.
        • Filippi A.R.
        • Fiandra C.
        • De Luca V.
        • Bartoncini S.
        • Vella D.
        • et al.
        Inclusion of heart substructures in the optimization process of volumetric modulated arc therapy techniques may reduce the risk of heart disease in Hodgkin’s lymphoma patients.
        Radiother Oncol. 2019; 138: 52-58https://doi.org/10.1016/j.radonc.2019.05.009
        • Gagliardi G.
        • Constine L.S.
        • Moiseenko V.
        • Correa C.
        • Pierce L.J.
        • Allen A.M.
        • et al.
        Radiation dose-volume effects in the heart.
        Int J Radiat Oncol Biol Phys. 2010; 76: S77-S85https://doi.org/10.1016/j.ijrobp.2009.04.093