Original Research Article| Volume 21, P18-23, January 2022

Ok

# Using multi-centre data to train and validate a knowledge-based model for planning radiotherapy of the head and neck

Open AccessPublished:January 25, 2022

## Abstract

### Background and purpose

Knowledge-based radiotherapy planning models have been shown to reduce healthy tissue dose and optimisation times, with larger training databases delivering greater robustness. We propose a method of combining knowledge-based models from multiple centres to create a ‘super-model’ using their collective patient libraries, thereby increasing the breadth of training knowledge.

### Materials and methods

A head and neck super-model containing 207 patient datasets was created by merging the data libraries of three centres. Validation was performed on 30 independent datasets during which optimiser parameters were tuned to deliver the optimal set of model template objectives. The super-model was tested on a further 40 unseen patients from four radiotherapy centres, including one centre external to the training process. The generated plans were assessed using established plan evaluation criteria.

### Results

The super-model generated plans that surpassed the dose objectives for all patients with single optimisations in an average time of 10 min. Healthy tissue sparing was significantly improved over manual planning, with dose reductions to parotid of 4.7 ± 2.1 Gy, spinal cord of 3.3 ± 0.9 Gy and brainstem of 2.9 ± 1.7 Gy. Target coverage met the established constraints but was marginally reduced compared with clinical plans.

### Conclusions

Three centres successfully merged patient libraries to create a super-model capable of generating plans that met plan evaluation criteria for head and neck patients with improvements in healthy tissue sparing. The findings indicate that the super-model could improve head and neck planning quality, efficiency and consistency across radiotherapy centres.

## 1. Introduction

Radiotherapy treatment planning methods are currently based on the skill of the planner and subjective trial and error optimisations. This means that plan quality can be inconsistent and potentially sub-optimal. As the complexity of treatment planning increases with the implementation of intensity-modulated radiation therapy (IMRT) and advances in machine capability, more focus can be placed on sparing organs at risk (OARs) to minimise toxicity, but this creates greater planning inconsistency between centres and planners due to variable protocols and techniques [
• Tol J.P.
• Delaney A.R.
• Dahele M.
• Slotman B.J.
• Verbakel W.
Evaluation of a Knowledge-Based Planning Solution for Head and Neck Cancer.
,
• Nwankwo O.
• Mekdash H.
• Sihono D.S.K.
• et al.
Knowledge-based radiation therapy (KBRT) treatment planning versus planning by experts: validation of a KBRT algorithm for prostate cancer treatment planning.
]. With increasingly challenging cases, departments are at risk of spending too much time and resource to produce plans that are not necessarily optimal or consistent in quality [
• Fogliata A.
• Reggiori G.
• Stravato A.
• et al.
RapidPlan head and neck model: the objectives and possible clinical benefit.
]. Knowledge-based planning (KBP) models have successfully been implemented clinically in many centres for multiple treatment sites and shown to reduce optimisation time and OAR doses [
• Tol J.P.
• Delaney A.R.
• Dahele M.
• Slotman B.J.
• Verbakel W.
Evaluation of a Knowledge-Based Planning Solution for Head and Neck Cancer.
,
• Ma C.
• Huang F.
Assessment of a knowledge-based RapidPlan model for patients with postoperative cervical cancer.
,
• Good D.
• Lo J.
• Lee R.W.
• Wu J.Q.
• Yin F.
• Das S.
A knowledge-based approach to improving and homogenizing intensity modulated radiation therapy planning quality among treatment centers: an example application to prostate cancer planning.
,
• Chang A.T.Y.
• Hung A.W.M.
• Cheung F.W.K.
• Lee M.C.H.
• Chan O.S.H.
• Philips H.
• et al.
Comparison of planning quality and efficiency between conventional and knowledge-based algorithms in nasopharyngeal cancer patients using intensity modulated radiation therapy.
,
• Fogliata A.
• Wang P.M.
• Belosi F.
• et al.
Assessment of a model based optimization engine for volumetric modulated arc therapy for patients with advanced hepatocellular cancer.
,
• Foy J.
• Marsh R.
• Ten Haken R.K.
• et al.
An analysis of knowledge-based planning for stereotactic body radiation therapy of the spine.
,
• Chatterjee
• et al.
Performance of knowledge-based radiation therapy planning for the glioblastoma disease site.
]. Additionally, consistent radiotherapy has been correlated with improved overall patient survival [
• Moore K.L.
• Brame R.S.
• Low D.A.
• et al.
Experience-based quality control of clinical intensity-modulated radiotherapy planning.
,
• Nelms B.E.
• Robinson G.
• Markham J.
• et al.
Variation in external beam treatment plan quality: an inter-institutional study of planners and planning systems.
,
• Peters L.J.
• O’Sullivan B.
• Giralt J.
• et al.
Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG0202.
], and KBP has been shown to reduce treatment planning variability [
• Hirashima H.
• Nakamura M.
• Mukumoto N.
• et al.
Reducing variability among treatment machines using knowledge-based planning for head and neck, pancreatic, and rectal cancer.
]. Studies have suggested that increasing the number of patients within the model library improves model performance [
• Tol J.P.
• Delaney A.R.
• Dahele M.
• Slotman B.J.
• Verbakel W.
Evaluation of a Knowledge-Based Planning Solution for Head and Neck Cancer.
] as it allows the model to be trained to account for a larger variety of patient geometries. Combining KBP models from different centres could therefore have the potential to improve the quality of radiotherapy treatments due to the larger dataset available for model training. Additionally, by combining models and distributing them internationally, a planning standard could be defined helping to improve international radiotherapy treatment consistency and quality.
This study investigates a platform, currently in prototype form, which allows multiple KBP models to be combined between centres to create a super-model using collective patient libraries, thus increasing the breadth of knowledge and statistics available to the model for training and therefore its applicability and potential success clinically. The aim of the merging platform is to make training models across centres faster, easier and more reproducible to enable the creation of super-models. These super-models will represent the knowledge and best practice of a consortium or network of centres and can be regularly updated based on expert plans to reflect current clinical practice. Sharing such models offers further opportunities, including standardisation of clinical trial planning protocols and quality assurance of treatment plans across centres. New, inexperienced centres could benefit from the expertise of others and gain confidence faster when commissioning new treatment sites. Additionally, the ability to combine data libraries could be particularly useful for rare cancers or techniques where individual centres may take a long time to accrue enough patients to train a model. This study aimed to generate and validate a head and neck super-model by combining the model libraries of three UK centres and training a KBP model on the merged dataset. Validation was performed on 10-patient evaluation groups at each of the three model-contributing centres, and model testing performed on 40 unseen independent patients from the same three centres plus an additional centre. Model performance was assessed using established plan-evaluation criteria detailed in Section 2.2. The overall aim was to assess the feasibility of creating a super-model using the merging platform and evaluate its clinical success in terms of standardisation, efficiency and quality in comparison with standard clinical planning techniques.

## 2. Materials and methods

### 2.2 Plan evaluation criteria

To enable comparison of treatment plans generated in multiple ways, plan evaluation criteria were established to serve as assessment points for the quality of the dose distribution. The criteria were based on the recommendations of the International Commission on Radiation Units (ICRU 83), which states that the near-minimum (D98%), near-maximum (D2%) and median (D50%) absorbed doses should be reported for each clinical plan [

ICRU 83. Prescribing, Recording, and Reporting Photon-Beam Intensity-Modulated Radiation Therapy (IMRT). J ICRU 2010;10(1).

]. The mean and maximum (0.1 cc) dose to individual OARs were collected. The OARs selected for comparison were the spinal cord, cord planning organ at risk volume (PRV), brainstem, brainstem PRV and the parotids, as these organs featured in the vast majority of clinical plans from all contributing centres. The final parameter chosen for comparison was the number of Monitor Units (MU) required to deliver each plan, as this provided a measure of plan complexity.

### 2.3 Model validation

The model was validated on a total of 30 patients; 10 from each of the centres contributing to the model library. Multiple optimiser objective combinations were trialled, alongside different parotid structure definitions, various NTO settings and MU objectives before the final super-model structure template was defined. Multiple iterations of model objectives were tested, starting with fully automated PTV and OAR objectives to minimise the user input, and ending with fixed objectives for the PTVs to assess the impact on plan quality. Separating the parotids into left and right, ipsilateral and contralateral and combining them into a single structure were all tested as combinations in the model template to assess which contouring technique provided the best OAR-sparing effect when running the model. Additionally, various NTO and MU objectives were tested with the aim of optimising plan quality and complexity. Each validation iteration involved testing the model on 10 patients, comparing with the previous model validation stage, and consulting with consortium members to evaluate the best model to proceed with for testing and eventually sharing with other centres.

### 2.4 Model testing

The merged super-model was tested on a total of 40 patients; 10 each from four different centres. Three of the testing centres had contributed patients to training the model, although all patients used for testing were independent of the model library and selected randomly. The super-model was run with a single optimisation and no manual input and the plans were normalised to 100% in the high dose target mean. The time required to generate the dose-volume histogram (DVH) estimates and optimise each plan was noted and the established plan evaluation criteria were extracted and compared with the clinical plans. Paired sample t-tests were used to assess whether mean OAR differences between super-model plans and clinical plans were statistically significant (p-value$<$0.05). Normality of the data was tested prior to the analysis with the Anderson–Darling test to ensure statistical validity. Plans were also visually assessed to compare plan quality.

## 3. Results

### 3.1 Model validation

The optimal objectives for the optimiser were found to be fully automated for OARs but fixed for PTVs, as detailed in Table 1. The original model produced plans with high PTV54 D50% doses and therefore an additional V50% objective was added to keep the values within tolerance. The model with combined parotids was also found to produce the lowest parotid doses and therefore the parotid geometries were combined for the final template. The addition of an MU objective was found to be unnecessary so no MU objective was fixed in the optimiser template.
Table 1A table outlining the optimiser objective template used in the merged Rapidplan super-model. Generated (Gen) objectives are fully automated line objectives created by the RapidPlan model.
StructureTypeVol (%)Dose (Gy)Priority
Upper067.0100
PTV 65Lower10065.0100
Lower9765.5100
Upper062.0100
PTV 60Lower10060.5100
Lower9761.0100
Upper057.090
PTV 54Upper5054.080
Lower10054100
Lower9755100
CordLineGenGenGen
Cord PRVLineGenGenGen
BrainstemLineGenGenGen
Brainstem PRVLineGenGenGen
ParotidsLineGenGenGen

### 3.2 Model testing

A comparison of plan quality for the clinical plans and super-model RapidPlans at each centre, measured using established plan evaluation criteria, is shown in Table 2.
Table 2A table outlining the mean ($±$SD) clinical and super-model RapidPlan evaluation results for the 40 patients included in the testing group (10 each from four centres). Super-model results marked with a * are statistically significantly different from their clinical counterpart (p$<$0.05).
ParametersClinicalSuper-model
D98$%$62.5 ± 0.861.9 ± 0.7*
PTV65D50$%$65.1 ± 0.165.2 ± 0.6
D2$%$66.6 ± 0.466.8 ± 0.5*
D98$%$58.1 ± 0.657.6 ± 1.3
PTV60D50$%$60.3 ± 0.260.6 ± 0.5
D2$%$62.9 ± 0.962.4 ± 0.6
D98$%$52.0 ± 0.751.7 ± 0.7*
PTV54D50$%$54.3 ± 0.254.8 ± 0.3*
D2$%$61.9 ± 1.462.0 ± 1.3
ParotidsMean30.9 ± 9.626.2 ± 9.0*
Cord0.1 cc38.9 ± 4.337.8 ± 2.2*
Mean23.6 ± 4.720.3 ± 3.1*
Cord PRV0.1 cc42.7 ± 4.342.4 ± 3.0
Mean23.6 ± 4.620.4 ± 3.1*
Brainstem0.1 cc33.2 ± 14.131.7 ± 13.3*
Mean11.1 ± 9.58.2 ± 5.5*
Brainstem PRV0.1 cc36.3 ± 15.036.2 ± 13.5
Mean10.1 ± 6.49.1 ± 5.6*
MUsTotal473 ± 70520 ± 54*
The coverage of the PTVs was slightly better in the clinical plans, although the RapidPlans were still clinically acceptable in this respect and met the established dose constraint of 98% of the volume receiving 95% of the prescribed dose. The maximum (0.1 cc) values for the cord and brainstem were significantly lower on average across the evaluation group for the RapidPlans. The mean dose for every OAR fell significantly, most notably for the parotids with a mean dose-sparing of 4.7 Gy. The mean values of the cord, cord PRV, brainstem and brainstem PRV dose also reduced significantly across the evaluation groups. This dose-sparing effect is shown graphically in Fig. 1, where the mean OAR values are shown for each individual centre alongside the mean PTV differences. The average optimisation and calculation time for the evaluation group RapidPlans was 10.1 min. The monitor units increased significantly for the Rapidplans suggesting increased plan complexity.
Fig. 2 shows the mean DVH plots for the primary PTV and two of the modelled OARs at one of the centres to highlight the dose differences across the entire volumes. The plots were created by calculating the average dose (Gy) at each relative volume point for the OARs for each of the 10 evaluated plans. The DVH graphs display a statistically significant reduction in dose across the cord and parotid volumes for the super-model RapidPlans (displayed in red) compared with the clinical plans (displayed in blue). They also show minimal changes between the primary PTV dose recorded for the RapidPlans and clinical plans, with the RapidPlan distributions being marginally less homogeneous.
Individual plans were compared visually to assess the differences between the clinical plans and super-model RapidPlans; an example is shown below. Fig. 3 shows a comparison of a clinical plan and generated merged RapidPlan for an oropharynx patient. It highlights one of the key differences between the clinical and RapidPlan generated plans that was a significant finding across all of the tested patients; better parotid sparing in the RapidPlan. The dose gradient is much steeper at the parotid-PTV interface which has reduced the parotid mean dose significantly from 34.3 Gy to 22.0 Gy. However, this has caused the PTV65Gy D98% coverage to reduce by 0.2 Gy in the RapidPlan, although it still meets the established objective.

## 4. Discussion

A head and neck super-model created by merging the RapidPlan data libraries of three centres was able to produce clinically acceptable plans in terms of established evaluation criteria for a total of 40 patients from four different centres, each contributing 10-patient evaluation groups. One of the testing centres did not contribute to the model building and training process, demonstrating the success of the model in an independent setting. The PTV coverage in the super-model plans was slightly lower than the clinical plans, although still met the clinical objectives, and the mean OAR doses were statistically significantly lower. It is a patient-specific decision by the clinician as to whether more PTV coverage is desirable or whether OAR sparing is more important in particular regions of the patient’s geometry. It is therefore difficult to state which is a better quality plan for each patient as they are both clinically acceptable in terms of statistics but one may be preferable to another for different clinicians. Determination of which plan is clinically better would be by clinicians; this will be the subject of further clinical trials.
With NHS England introducing a new 17-day treatment pathway for category one radiotherapy patients [

], it is becoming increasingly important to reduce the planning time for head and neck treatments. The super-model plans were generated in single optimisations with an average optimisation and planning time of 10 min. No clinical timing data was available, but a study which aimed to evaluate the machine and operator time required to treat head and neck patients identified the average treatment planning time to be 3 h and 8 min [
• Budach W.
• Bölke E.
• Fietkau R.
• et al.
Evaluation of time, attendance of medical staff, and resources during radiotherapy for head and neck cancer patients: the DEGRO-QUIRO trial.
]. This figure includes the pre-optimisation time required for PRV and optimisation contouring, beam placement and isocentre placement and so cannot be directly compared with the super-model optimisation time. However, even with a generous estimate of a pre-optimisation time of 1 h, using the super-model with a single optimisation still reduces the planning time by roughly two hours per patient. This is very significant for departments with busy workloads and addresses one of the challenges in meeting new more stringent NHS treatment pathways whilst simultaneously improving the consistency of plan quality.
Methods to boost the PTV coverage whilst maintaining good OAR sparing have begun, with testing including varying the NTO settings in the optimiser, and introducing a PTV boost volume and re-optimising the plan with a boost priority value. Both of these methods have been shown to improve PTV coverage, although not without consequence. Relaxing the NTO objective improves coverage but increases the normal tissue dose and boosting the PTV volume increases the MUs and the overall planning time. The super-model already produces plans with higher MUs, most likely due to the higher complexity needed to achieve significantly lower OAR doses, so any further increase would need to be considered with care. The model could be manipulated individually by each centre to achieve the clinical goals of each clinician, whilst still providing a standardised and good quality plan template with the initial optimisation. An MU objective could also be introduced on a centre-by-centre basis depending on specific QA requirements and pass criteria.
Concerning the patient data input into the model, 207 patients were used with a variety of plan sites including: oropharynx, hypopharynx, larynx and nasopharynx. Due to the rare nature of nasopharyngeal cancers, only 23 nasopharynx plans were available for use. Therefore the model data was limited for this type of tumour, reducing the accuracy of the model’s DVH prediction ability. Furthermore, due to the varying protocols and techniques used by different centres, not all of the OARs were present in each plan for all centres and consequently the model was unable to train a number of additional reported head and neck OARs. Only the OARs which were in use at all centres for all patients were included in the model and so, to achieve local objectives/tolerances, it may be necessary for individual centres to add objectives to the optimisation template to drive optimisation of dose to any additional structures of interest. With a larger data set of nasopharynx patient plans and more standardised contouring protocols, additional OARs could be trained, widening the model’s clinical use and reducing the need for manual adjustments at each centre. However, it could be beneficial for the model to remain in its primary format with fewer OARs since planning and reporting techniques vary so significantly between centres. This allows it to be a useful standard and potential building block for a larger group of patients.
To the best of the author’s knowledge, the prototype merging platform used in this study is the first of its kind in that it allows Rapidplan models to be combined without transferring and sending patient data between centres; it simply requires each centre to connect and upload its model to a portal. This ensures data protection laws are complied with and approvals are not necessary each time a new model is compiled. Another successful method to create a multi-centre model was investigated by Panettieri et al. [
• Panettieri V.
• Ball D.
• Chapman A.
• et al.
Development of a multicentre automated model to reduce planning variability in radiotherapy of prostate cancer.
] for prostate IMRT, but this required sending anonymised patient data between centres and then re-building and training the model from scratch, which lengthens the merging process and may be subject to a wider range of legal or regulatory restrictions. The accessibility of the merging platform opens the opportunities of the super-model to a wide set of users and allows centres with minimal experience or lack of data to use high-quality and well-tested super-models. Multi-centre validation of models has been common practice for a while [
• Hussein M.
• Currie S.
• et al.
OC-0185 A multi-centre knowledge-based treatment planning model for radiotherapy of cervical cancer.
,
• Kamima T.
• Ueda Y.
• Fukunaga J.
• et al.
Multi-institutional evaluation of knowledge-based planning performance of volumetric modulated arc therapy (VMAT) for head and neck cancer.
,
• Fiandra C.
• Rossi L.
• Alparone A.
• et al.
Automatic genetic planning for volumetric modulated arc therapy: A large multi-centre validation for prostate cancer.
], and has shown that combining experience between centres can improve model performance and standardisation of radiotherapy [
• Moore K.L.
• Brame R.S.
• Low D.A.
• et al.
Experience-based quality control of clinical intensity-modulated radiotherapy planning.
,
• Nelms B.E.
• Robinson G.
• Markham J.
• et al.
Variation in external beam treatment plan quality: an inter-institutional study of planners and planning systems.
,
• Peters L.J.
• O’Sullivan B.
• Giralt J.
• et al.
Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG0202.
]. Therefore, the ease with which multiple centres can contribute to and influence models plays an important role in improving radiotherapy treatment.
This study investigates the head and neck region but this method of merging models could be applied to multiple treatment sites, as demonstrated by Panettieri et al. [
• Panettieri V.
• Ball D.
• Chapman A.
• et al.
Development of a multicentre automated model to reduce planning variability in radiotherapy of prostate cancer.
] for the prostate, and prescriptions and help to standardise a wide range of radiotherapy treatments. One of the advantages of the platform is that it is very quick and simple to re-merge and therefore update super-models, meaning they can be continuously improved and re-distributed in line with changes in clinical practice and changing prescriptions. Rapidplan has been shown to adapt well to ranging prescriptions [
• Shepherd M.
• Bromley R.
• Ma Stevens
• et al.
Developing knowledge-based planning for gynaecological and rectal cancers: a clinical validation of RapidPlan.
], when given an appropriate data library, which is more feasible with larger numbers of centres able to contribute data via the merging portal.
Additionally, merging models could be useful tool for clinical trials where standardisation of care is pivotal, and also for quality assurance (QA) purposes where the model could be used to ensure a predetermined planning quality is being maintained. Previous studies have used single institution KBP models as QA tools for multi-institutional clinical trials and found plan quality to improve with the process [
• Kavanaugh J.
• Holler S.
• DeWees T.
• et al.
Multi-Institutional Validation of a Knowledge-Based Planning Model for Patients Enrolled in RTOG 0617: Implications for Plan Quality Controls in Cooperative Group Trials.
,
• Duffy S.R.
• Zheng Y.
• Muenkel J.
• et al.
Refining complex re-irradiation dosimetry through feasibility benchmarking and analysis for informed treatment planning.
]. Super-models and distributing platforms could enable QA to happen on a larger scale.
The merging platform allowed three centres to successfully merge their patient libraries to create a super-model capable of generating plans that met the plan evaluation criteria for head and neck patients with improvements in OAR sparing. The super-model reduced planning time and provided a solution for improving the consistency of the head and neck planning process. The primary indications are that the super-model will help to improve head and neck planning efficiency and consistency across centres.

## Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Two of the authors are employees of Varian Medical Systems – a Healthineers company.

## References

• Tol J.P.
• Delaney A.R.
• Dahele M.
• Slotman B.J.
• Verbakel W.
Evaluation of a Knowledge-Based Planning Solution for Head and Neck Cancer.
Int J Radiat Oncol Biol Phys. 2015; 1: 612-620https://doi.org/10.1016/j.ijrobp.2014.11.014
• Nwankwo O.
• Mekdash H.
• Sihono D.S.K.
• et al.
Knowledge-based radiation therapy (KBRT) treatment planning versus planning by experts: validation of a KBRT algorithm for prostate cancer treatment planning.
• Fogliata A.
• Reggiori G.
• Stravato A.
• et al.
RapidPlan head and neck model: the objectives and possible clinical benefit.
• Ma C.
• Huang F.
Assessment of a knowledge-based RapidPlan model for patients with postoperative cervical cancer.
Precision Radiat Oncol. 2017; 1: 102-207https://doi.org/10.1002/pro6.23
• Good D.
• Lo J.
• Lee R.W.
• Wu J.Q.
• Yin F.
• Das S.
A knowledge-based approach to improving and homogenizing intensity modulated radiation therapy planning quality among treatment centers: an example application to prostate cancer planning.
Int J Radiat Oncol Biol Phys. 2013; 87: 176-181https://doi.org/10.1016/j.ijrobp.2013.03.015
• Chang A.T.Y.
• Hung A.W.M.
• Cheung F.W.K.
• Lee M.C.H.
• Chan O.S.H.
• Philips H.
• et al.
Comparison of planning quality and efficiency between conventional and knowledge-based algorithms in nasopharyngeal cancer patients using intensity modulated radiation therapy.
Int J Radiat Oncol Biol Phys. 2016; 95: 981-990https://doi.org/10.1016/j.ijrobp.2016.02.017
• Fogliata A.
• Wang P.M.
• Belosi F.
• et al.
Assessment of a model based optimization engine for volumetric modulated arc therapy for patients with advanced hepatocellular cancer.
• Foy J.
• Marsh R.
• Ten Haken R.K.
• et al.
An analysis of knowledge-based planning for stereotactic body radiation therapy of the spine.
Pract Radiat Oncol. 2017; 7: e355-e360https://doi.org/10.1016/j.prro.2017.02.007
• Chatterjee
• et al.
Performance of knowledge-based radiation therapy planning for the glioblastoma disease site.
Int J Radiat Oncol Biol Phys. 2017; 99: 1021-1028https://doi.org/10.1016/j.ijrobp.2017.07.012
• Moore K.L.
• Brame R.S.
• Low D.A.
• et al.
Experience-based quality control of clinical intensity-modulated radiotherapy planning.
Int J Radiat Oncol Biol Phys. 2011; 81 (https://10.1016/j.ijrobp.2010.11.030): 545-551
• Nelms B.E.
• Robinson G.
• Markham J.
• et al.
Variation in external beam treatment plan quality: an inter-institutional study of planners and planning systems.
Pract Radiat Oncol. 2012; 2 (https://10.1016/j.prro.2011.11.012): 296-305
• Peters L.J.
• O’Sullivan B.
• Giralt J.
• et al.
Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG0202.
J Clin Onc. 2010; 28 (https://10.1200/JCO.2009.27.4498): 2996-3001
• Hirashima H.
• Nakamura M.
• Mukumoto N.
• et al.
Reducing variability among treatment machines using knowledge-based planning for head and neck, pancreatic, and rectal cancer.
J Appl Clin Med Phys. 2021; 22 (https://10.1002/acm2.13316): 245-254
1. ICRU 83. Prescribing, Recording, and Reporting Photon-Beam Intensity-Modulated Radiation Therapy (IMRT). J ICRU 2010;10(1).

• Budach W.
• Bölke E.
• Fietkau R.
• et al.
Evaluation of time, attendance of medical staff, and resources during radiotherapy for head and neck cancer patients: the DEGRO-QUIRO trial.
Strahlenther Onkol. 2011; 187: 449-460https://doi.org/10.1007/s00066-011-2273-z
• Panettieri V.
• Ball D.
• Chapman A.
• et al.
Development of a multicentre automated model to reduce planning variability in radiotherapy of prostate cancer.
Phys Imaging Radiat Oncol. 2019; 11 (https://10.1016/j.phro.2019.07.005): 34-40
• Hussein M.
• Currie S.
• et al.
OC-0185 A multi-centre knowledge-based treatment planning model for radiotherapy of cervical cancer.
• Kamima T.
• Ueda Y.
• Fukunaga J.
• et al.
Multi-institutional evaluation of knowledge-based planning performance of volumetric modulated arc therapy (VMAT) for head and neck cancer.
Physica Medica. 2019; 64: 174-181https://doi.org/10.1016/j.ejmp.2019.07.004
• Fiandra C.
• Rossi L.
• Alparone A.
• et al.
Automatic genetic planning for volumetric modulated arc therapy: A large multi-centre validation for prostate cancer.
• Shepherd M.
• Bromley R.
• Ma Stevens
• et al.
Developing knowledge-based planning for gynaecological and rectal cancers: a clinical validation of RapidPlan.
J Med Rad Sci. 2010; 67: 217-224https://doi.org/10.1002/jmrs.396
• Kavanaugh J.
• Holler S.
• DeWees T.
• et al.
Multi-Institutional Validation of a Knowledge-Based Planning Model for Patients Enrolled in RTOG 0617: Implications for Plan Quality Controls in Cooperative Group Trials.
Pract Radiat Oncol. 2019; 9: 218-227https://doi.org/10.1016/j.prro.2018.11.007
• Duffy S.R.
• Zheng Y.
• Muenkel J.
• et al.
Refining complex re-irradiation dosimetry through feasibility benchmarking and analysis for informed treatment planning.
J Appl Clin Med Phys. 2020; 21 (https://10.1002/acm2.13102): 263-271