Adaptive radiotherapy (ART) planning and delivery strategies are often a prerequisite for accurate dose delivery for several tumour sites influenced by inter-fractional anatomical changes. This includes tumours in a number of sites such as the head-and-neck, as this anatomical region can experience nasal cavity filling and weight loss [
[1]
]; in the lung, where tumour baseline shifts and atelectasis can occur [[2]
]; and in the cervix, where different levels of bladder or rectum filling can impact the target coverage [[3]
].ART relies heavily on the accessibility and quality of in-room imaging. Different modalities have been suggested for use in adaptive workflows, including magnetic resonance imaging (MRI) [
[4]
], computed tomography (CT) [[5]
], and cone beam CT (CBCT) [[3]
]. The advantage of using CBCT for this purpose is its wide availability; many treatment rooms for both photon and proton therapy are equipped with CBCT scanners, and daily or weekly CBCT scans are often acquired for verification of patient positioning. These verification images are in general used to visibly assess if anatomical changes have occurred that warrant plan adaptation. If a plan adaptation is flagged, the patient can be referred for a re-planning CT scan; however, dose calculations directly on the CBCT scans have also been investigated [6
, 7
, 8
].The image quality of CBCT scans is typically much lower than for CT scans which can reduce the accuracy of CBCT-based dose calculation [
9
, 10
]. The artefacts in CBCT scans and resulting unreliable CT numbers in CBCTs are especially detrimental in proton therapy due to the steep fall-off of the proton depth-dose curve which requires a high accuracy of the CT number to stopping power ratio conversion to estimate the proton range [[11]
]. Several techniques have been suggested for artefact reduction or correction in CBCT for both photon and proton therapy. These include look-up table-based approaches [[12]
], deformable imaging registration (DIR) of the treatment planning CT (pCT) to the CBCT [13
, 14
], scatter correction methods using a prior based on the pCT [8
, 15
], methods which model the scatter contribution of the measured projections [[16]
], or Monte Carlo-based approaches [[17]
]. Among these methods there are approaches working at both image [12
, 13
, 14
] or projection level [8
, 15
, 16
, 17
]. More recently, novel approaches for CBCT corrections based on neural networks and deep learning have been proposed [11
, 18
, - Landry G.
- Hansen D.
- Kamp F.
- Li M.
- Hoyle B.
- Weller J.
- et al.
Comparing Unet training with three different datasets to correct CBCT images for prostate radiotherapy dose calculations.
Phys Med Biol. 2019; 64035011https://doi.org/10.1088/1361-6560/aaf496
19
]. Different network architectures have been applied; among the most successful approaches are cycle-consistent generative adversarial networks (cycle-GANs) [20
, - Liang X.
- Chen L.
- Nguyen D.
- Zhou Z.
- Gu X.
- Yang M.
- et al.
Generating synthesized computed tomography (CT) from cone-beam computed tomography (CBCT) using CycleGAN for adaptive radiation therapy.
Phys Med Biol. 2019; 64125002https://doi.org/10.1088/1361-6560/ab22f9
21
, - Kurz C.
- Maspero M.
- Savenije M.H.F.
- Landry G.
- Kamp F.
- Pinto M.
- et al.
CBCT correction using a cycle-consistent generative adversarial network and unpaired training to enable photon and proton dose calculation.
Phys Med Biol. 2019; 64225004https://doi.org/10.1088/1361-6560/ab4d8c
22
]. The aim of CBCT correction based on cycleGANs is to create a so-called synthetic CT (sCT) from the CBCT, where the sCT should have an appearance, CT number accuracy, and image consistency of the same quality as the pCT. This is the research avenue of the study by Maspero et al. recently published in this journal [- Liu Y.
- Lei Y.
- Wang T.
- Fu Y.
- Tang X.
- Curran W.J.
- et al.
CBCT-based synthetic CT generation using deep-attention cycleGAN for pancreatic adaptive radiotherapy.
Med Phys. 2020; https://doi.org/10.1002/mp.14121
[23]
]. Maspero et al. applied a cycleGAN to correct CBCTs for breast, lung and head-and-neck cancer patients to enable photon dose calculations on the corrected CBCTs, the sCTs [[23]
].An advantage of deep learning approaches compared to the physics based approaches utilizing, e.g., Monte Carlo simulations, is that deep learning is fast, and it can typically correct a whole CBCT image stack within seconds [
[23]
]. The training of well-functioning deep learning models, however, requires a lot of time, training data and extensive computational power [[24]
]. The computer optimizes the neural network parameters, learning the similarities and differences between the two image sets – pCTs and CBCTs in this case – and how to convert one into the other. For cycleGANs, or generative adversarial networks (GANs) in general, this training process can be seen as a competition between two conflicting networks, where one tries to create a sCT from a CBCT with an image quality high enough that it resembles a regular CT, while the other aims at being able to distinguish between CT (real) and sCT (fake) images [[25]
]. To improve the quality of the mapping for cycleGANs, the inverse transformation, in this case from a CT to a synthetic CBCT (sCBCT), is also created. The consistency over the cycle is then constrained such that the conversion from a CBCT to a CT and back to a CBCT should result in a CBCT, which is virtually the same as the original CBCT, and the same for the conversion from CT to CBCT and to CT again. This cycle consistency enables training based on unpaired data. This means that in each step of the training process, the algorithm is presented with CT and CBCT image slices but not necessarily from the same patient or the same anatomical location [[23]
], as the algorithm is based on feature extraction rather than on creating a one-to-one correspondence. The correspondence is instead ensured by the constraint of regaining the same image after a full cycle of the process [[25]
]. This simplifies the collection of training data since images can be included for patients where either the CT or CBCT is missing, or where large anatomical changes are seen between the CT and CBCT, which can create problems for other CBCT correction methods based on e.g. DIR.Deep learning has been used in many fields already, and image processing tasks may be especially suitable to be tackled by artificial intelligence. A clear advantage is the versatility, as approaches already extensively used in one discipline can be easily transferred and reused in another field of application. Maspero et al. successfully apply a network to correct CBCTs which was originally developed for image processing tasks very remote from the medical physics world [
[23]
]. The cycleGAN network was originally designed to create naturally looking photographs from Van Gogh paintings, or aerial photos from Google map images [[25]
]. Even though the two set of tasks seem to be very unrelated – correcting CBCT images by utilizing high quality pCTs and converting paintings to photographs – the common trait is that both tasks involve image-to-image conversions, i.e., mapping from one image domain (either CBCTs or paintings) to another (CTs or photographs). Deep learning is agnostic to the underlying characteristics; it tackles the two tasks in exactly the same manner: It learns the features of the two domains based on a large data set of training examples, and how to connect the features from the two domains. When the network after the end of the training stage is presented with a new, unseen example, it knows how to relate this image to an image of the other domain [[25]
].The novelty in the paper by Maspero et al. is the successful training of a single deep learning network to correct CBCT images of different anatomical regions: breast, lung and head-and-neck [
[23]
]. The single network was compared to three networks trained individually for each anatomical site in terms of image accuracy for the generated sCTs. The image similarity, measured as the mean absolute error and the mean error, between the sCT and the rescan CT (rCT), was comparable between the results obtained applying the networks trained only on a single anatomical site and the network trained for the three sites together [[23]
]. This result has very favourable consequences for clinical implementation, as the very cumbersome and time-consuming task of training the network can be done only once and then used for all patients and indications. And maybe more importantly, only one model should be applied, which eliminates the risk of using the wrong model in a busy clinical situation.The fact that a single network can be used for such diverse anatomical sites as the breast, lung and head-and-neck might also highlight the fact that deep learning models are often extracting features from the data with no (or little) physical meaning rather than building human interpretable models. There are both risks and benefits in using deep learning [
[26]
], and this debate is of high importance. But likely CBCT correction is a field where we can safely, though never uncritically, welcome artificial intelligence. Here the deep learning model is not used to diagnose or to predict the survival of the patient [[27]
], but instead to improve the CBCT image contrast and consistency [[23]
].After the training stage, the cycleGAN model by Maspero et al. can generate a sCT from a CBCT within 10 s on a GPU and 40 s on a CPU [
[23]
], whereby image conversion is feasible while the patient is on the treatment table. The speed is one of the common and very important advantages of deep learning. Another advantage of their model is that it can conserve the anatomy observed in the CBCT images [[23]
]. Unlike methods that rely on DIR to transform the CBCT to the pCT, cycleGAN uses only the CBCT image as the input after training. For one lung test patient that presented atelectasis on the CBCT, but not on the pCT, the atelectasis was correctly represented also in the sCT [[23]
]; whereas methods relying on DIR to the pCT can lead to anatomical misrepresentation [[11]
]. The improved image quality of the sCT, compared to the original CBCT, also allows for accurate dose re-computations with clinical acceptable voxel-vise dose differences and gamma-pass rates for photon-based radiotherapy plans [[23]
]. It is known that proton dose calculations are more susceptible to subtle difference than photon radiotherapy and larger dose difference can be seen [11
, 18
]. It would therefore be of interest to see how well the method of Maspero et al. would perform in proton therapy.- Landry G.
- Hansen D.
- Kamp F.
- Li M.
- Hoyle B.
- Weller J.
- et al.
Comparing Unet training with three different datasets to correct CBCT images for prostate radiotherapy dose calculations.
Phys Med Biol. 2019; 64035011https://doi.org/10.1088/1361-6560/aaf496
To conclude, with the improvement in CT number accuracy for CBCT images that can be gained by deep learning approaches, such as the ones presented by Maspero et al. [
[23]
], we get closer to the goal of online ART, where the treatment plan can be recalculated while the patient is lying on the treatment couch. The next step towards this goal may be automated segmentation [28
, 29
] and fast dose calculations [30
, - Nguyen D.
- Jia X.
- Sher D.
- Lin M.-H.
- Iqbal Z.
- Liu H.
- et al.
3D radiotherapy dose prediction on head and neck cancer patients with a hierarchically densely connected U-net deep learning architecture.
Phys Med Biol. 2019; 64065020https://doi.org/10.1088/1361-6560/ab039b
31
], and deep learning might again be crucial to achieve the final goal of fully automated, adaptive re-planning.Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
This work was partially supported by MSK Cancer Center Support Grant/Core Grant from the National Institute of Health (NIH), United States (P30 CA008748).
References
- Dose accumulation to assess the validity of treatment plans with reduced margins in radiotherapy of head and neck cancer.Phys Imag Radiat Oncol. 2020; 14: 53-60https://doi.org/10.1016/j.phro.2020.05.004
- Automatic selection of lung cancer patients for adaptive radiotherapy using cone-beam CT imaging.Phys Imag Radiat Oncol. 2017; 1: 21-27https://doi.org/10.1016/j.phro.2017.02.005
- Cone beam computed tomography-based monitoring and management of target and organ motion during external beam radiotherapy in cervical cancer.Phys Imag Radiat Oncol. 2019; 9: 14-20https://doi.org/10.1016/j.phro.2018.12.002
- Clinical implementation of magnetic resonance imaging guided adaptive radiotherapy for localized prostate cancer.Phys Imag Radiat Oncol. 2019; 9: 69-76https://doi.org/10.1016/j.phro.2019.02.002
- On-line dose-guidance to account for inter-fractional motion during proton therapy.Phys Imag Radiat Oncol. 2019; 9: 7-13https://doi.org/10.1016/j.phro.2018.11.009
- Cone beam CT based dose calculation in the thorax region.Phys Imag Radiat Oncol. 2018; 7: 45-50https://doi.org/10.1016/j.phro.2018.09.001
- Feasibility of improving cone-beam CT number consistency using a scatter correction algorithm.J Appl Clin Med Phys. 2013; 14: 167-176https://doi.org/10.1120/jacmp.v14i6.4346
- Proton dose calculation on scatter-corrected CBCT image: feasibility study for adaptive proton therapy.Med Phys. 2015; 42: 4449-4459https://doi.org/10.1118/1.4923179
- Accuracy of dose calculation based on artefact corrected Cone Beam CT images of lung cancer patients.Phys Imag Radiat Oncol. 2017; 1: 6-11https://doi.org/10.1016/j.phro.2016.11.001
- Evaluating the impact of cone-beam computed tomography scatter mitigation strategies on radiotherapy dose calculation accuracy.Phys Imag Radiat Oncol. 2019; 10: 35-40https://doi.org/10.1016/j.phro.2019.04.001
- ScatterNet: A convolutional neural network for cone-beam CT intensity correction.Med Phys. 2018; 45: 4916-4926https://doi.org/10.1002/mp.13175
- Comparing cone-beam CT intensity correction methods for dose recalculation in adaptive intensity-modulated photon and proton therapy for head and neck cancer.Acta Oncol. 2015; 54: 1651-1657https://doi.org/10.3109/0284186X.2015.1061206
- First clinical investigation of cone beam computed tomography and deformable registration for adaptive proton therapy for lung cancer.Int J Radiat Oncol. 2016; 95: 549-559https://doi.org/10.1016/j.ijrobp.2016.01.055
- Investigating CT to CBCT image registration for head and neck proton therapy as a tool for daily dose recalculation: CT to CBCT dir for H&N particle therapy.Med Phys. 2015; 42: 1354-1366https://doi.org/10.1118/1.4908223
- Shading correction for on-board cone-beam CT in radiation therapy using planning MDCT images: MDCT-based shading correction for CBCT.Med Phys. 2010; 37: 5395-5406https://doi.org/10.1118/1.3483260
- A model-based scatter artifacts correction for cone beam CT.Med Phys. 2016; 43: 1736https://doi.org/10.1118/1.4943796
- Efficient scatter distribution estimation and correction in CBCT using concurrent Monte Carlo fitting: efficient scatter distribution estimation and correction in CBCT.Med Phys. 2014; 42: 54-68https://doi.org/10.1118/1.4903260
- Comparing Unet training with three different datasets to correct CBCT images for prostate radiotherapy dose calculations.Phys Med Biol. 2019; 64035011https://doi.org/10.1088/1361-6560/aaf496
- Scatter artifacts removal using learning-based method for CBCT in IGRT system.IEEE Access. 2018; 6: 78031-78037https://doi.org/10.1109/ACCESS.2018.2884704
- Generating synthesized computed tomography (CT) from cone-beam computed tomography (CBCT) using CycleGAN for adaptive radiation therapy.Phys Med Biol. 2019; 64125002https://doi.org/10.1088/1361-6560/ab22f9
- CBCT correction using a cycle-consistent generative adversarial network and unpaired training to enable photon and proton dose calculation.Phys Med Biol. 2019; 64225004https://doi.org/10.1088/1361-6560/ab4d8c
- CBCT-based synthetic CT generation using deep-attention cycleGAN for pancreatic adaptive radiotherapy.Med Phys. 2020; https://doi.org/10.1002/mp.14121
- A single neural network for cone-beam computed tomography-based radiotherapy of head-and-neck, lung and breast cancer.Phys Imag Radiat Oncol. 2020; 14: 24-31https://doi.org/10.1016/j.phro.2020.04.002
- Patch-based generative adversarial neural network models for head and neck MR-only planning.Med Phys. 2020; 47: 626-642https://doi.org/10.1002/mp.13927
Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. 2017 IEEE Int. Conf. Comput. Vis. ICCV, 2017, p. 2242–51. https://doi.org/10.1109/ICCV.2017.244.
- Clinical implementation of AI technologies will require interpretable AI models.Med Phys. 2020; 47: 1-4https://doi.org/10.1002/mp.13891
Ahmad MA, Teredesai A, Eckert C. Interpretable machine learning in healthcare. 2018 IEEE Int. Conf. Healthc. Inform. ICHI, 2018, p. 447–447. https://doi.org/10.1109/ICHI.2018.00095.
- Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis.Phys Imag Radiat Oncol. 2020; 14: 61-66https://doi.org/10.1016/j.phro.2020.05.009
- Improving automatic delineation for head and neck organs at risk by deep learning contouring.Radiother Oncol. 2020; 142: 115-123https://doi.org/10.1016/j.radonc.2019.09.022
- 3D radiotherapy dose prediction on head and neck cancer patients with a hierarchically densely connected U-net deep learning architecture.Phys Med Biol. 2019; 64065020https://doi.org/10.1088/1361-6560/ab039b
- A feasibility study on an automated method to generate patient-specific dose distributions for radiotherapy using deep learning.Med Phys. 2019; 46: 56-64https://doi.org/10.1002/mp.13262
Article info
Publication history
Published online: August 12, 2020
Identification
Copyright
© 2020 The Authors. Published by Elsevier B.V. on behalf of European Society of Radiotherapy & Oncology.
User license
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0) | How you can reuse 
Elsevier's open access license policy
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0)
Permitted
For non-commercial purposes:
- Read, print & download
- Redistribute or republish the final article
- Text & data mine
- Translate the article (private use only, not for distribution)
- Reuse portions or extracts from the article in other works
Not Permitted
- Sell or re-use for commercial purposes
- Distribute translations or adaptations of the article
Elsevier's open access license policy
