Radiobostt project has the primary objective of investigating the performance of radiomics-based machine learning for classification of bone and soft tissue tumors in both diagnosis-related tasks, such as discrimination of tumor type and grading, and prognosis-related tasks, such as prediction of response to neoadjuvant therapy, local recurrence or distant metastasis, and survival.
The secondary objectives are:
• To identify reliable and informative radiomic features of bone and soft tissue tumors to be
included in machine-learning classifiers;
• To compare the performance of radiomics-based machine-learning classifiers with
experienced and specialized radiologists;
• To allow sharing of experiments and machine-learning algorithms for classification of bone
and soft tissue tumors with other physicians and scientists remotely.
Methodology and project design
This project will consist of observational studies with both prospective and retrospective designs. Particularly, studies focused on prediction of diagnosis – such as benign vs. malignant tumor
discrimination and tumor grading – will be conducted retrospectively, as this design allows including larger numbers of patients with uncommon diseases, such as bone or soft tissue tumors
with imaging data already available. Studies focused on prediction of prognosis – such as response to neoadjuvant therapy, local recurrence or distant metastasis, and survival – will be conducted
prospectively, as this design allows better monitoring of patients during follow-up based on updated criteria established in the literature. Namely, according to the current guidelines on the management of bone and soft tissue sarcomas, patients treated for high-grade sarcoma are followed with clinical examination and local and chest imaging every 3-4 months for the first 2-3 years, then every 6 months up to the fifth year and annually thereafter. In patients treated for low-grade sarcoma, the frequency of follow-up visits is lower, i.e. every 6 months for the first 2-3 years and then annually.
Inclusion/exclusion criteria and reference standard
Institutional databases from the orthopedic surgery, pathology and radiology departments will be used for patients’ recruitment. Inclusion criteria will be:
• Patients with bone or soft tissue tumors that undergo surgery such as marginal excision or wide resection, either with or without neoadjuvant therapy, and are proven by the
pathological assessment of the surgical specimen;
• Patients with benign bone or soft tissue tumors that are managed conservatively and are proven by no change in characteristics, shape and size at imaging follow-up adequately
timed according to the tumor type;
• Preoperative imaging studies, including unenhanced/contrast-enhanced MRI and CT, available for radiomic analysis.
Patients not fulfilling the above criteria or with images affected by artifacts will be excluded. In studies focused on diagnosis-related tasks – such as benign vs. malignant discrimination,
tumor grading or typing – the reference standard will be histology based on the assessment of the surgical specimen in all cases excepting untreated benign lesions with unequivocal and unchanged
imaging findings at an adequately timed follow-up. In studies focused on the prediction of local or distant recurrence and response to therapy, the diagnosis will be based on histology (when available for ethical reasons) or consistent imaging findings, as the reference standard. In studies dealing with survival prediction, survival will be assessed based on follow-up.
Radiomic pipeline for classifier development
After a preliminary reliability analysis, the radiomic pipeline will include sequential steps as follows:
• Image segmentation will be performed by experienced radiologists using the approach that provides the highest number of informative and reliable radiomic features according to the
preliminary reliability analysis.
• Image preprocessing will deal with dependencies on some image parameters, such as the size of the pixels or voxels, the number of gray levels, the range of gray level values and
signal intensity non-uniformities, which are mostly related to different acquisition techniques. Preprocessing will consist of resampling to an isotropic pixel or voxel, intensity
normalization and discretization.
• Feature extraction will enable the extraction of several radiomic features, such as first-order, shape-based and textural (gray-level co-occurrence [GLCM], gray-level size zone
[GLSZM], gray-level run length [GLRLM] and gray-level dependence [GLDM]) features from original, Laplacian of Gaussian-filtered and wavelet-transformed images.
• Feature selection will allow dimensionality reduction, i.e. bringing the very large number of initial features to much lower levels. This process aims at excluding redundant features and
identifying the most informative ones in order to optimize the classification model performance.
• Classification models will be developed using machine-learning algorithms, such as knearest neighbors, naive Bayes, support vector machine, decision tree, random forest and
neural networks, variably combined with ensemble techniques like adaptive boosting (Adaboost) and bootstrap aggregation. These classifiers will enable various classification
tasks according to the clinical problem to be answered, such as discrimination of different tumor types, benign vs. malignant tumor discrimination, tumor grading, prediction of
response to neoadjuvant therapy, local recurrence or distant metastasis, and survival. The classifiers will be trained on labeled data (training set) from the PI’s institution. Therefore,
their performance will be validated using independent data sets from the external institutions involved in this project (external test set).
Radiomic analysis will be performed using open-source software, such as ITK-SNAP for image segmentation, PyRadiomics or 3D Slicer with HeterogeneityCAD (hCAD) module for feature extraction and WEKA for data mining.
Data analysis
Data analysis will be performed after a qualitative image assessment. Reliability of the radiomics features will be assessed using the intraclass correlation coefficient (ICC). Good-to-excellent (ICC≥0.75) reliability will define feature stability. Performance evaluation of the classifications will be determined using the area under the receiver operating characteristic curve (AUC) or other specific metrics, including: F-score, i.e. the harmonic average of the precision (also known as positive predictive value) and recall (also known as sensitivity), ranging from 0 to 1 (perfect accuracy); Matthews correlation coefficient, i.e. a measure of the quality of binary classifications in machine learning, ranging from + 1 (perfect prediction) to 0 (average random prediction) and − 1 (inverse prediction); area under the precision-recall curve, i.e. an alternative to AUC, which is more informative for unbalanced classes. The diagnostic performance of the radiomic classifiers and the radiologists will be compared using conventional statistical methods.
Data sharing
Anonymized datasets, experiments and validated machine-learning models for classification of bone and soft tissue tumors on medical images will be shared remotely with other physicians and
scientists working in this field. The present platform, with reserved and authenticated access control, will be used for this aim.
