Original Coverage & Source Attribution: bioengineer.org
A comprehensive systematic review evaluating five years of predictive artificial intelligence research in neurosurgery has revealed that while computational models demonstrate consistently high discriminative accuracy, pervasive deficiencies in statistical calibration and external validation continue to hinder safe clinical translation. Predictive modeling using artificial intelligence has expanded rapidly across surgical subdisciplines, supported by the convergence of large electronic health record datasets, advanced neuroimaging pipelines, and maturing machine learning architectures. These computational systems are intended to support complex neurosurgical practice by improving preoperative risk stratification, patient-specific outcome forecasting, surgical planning, and clinical decision-making. However, a rigorous assessment of the published literature indicates that the technical sophistication of an algorithm does not guarantee clinical reliability, as models frequently suffer from significant performance degradation when applied to independent patient populations and routinely omit evaluations of probability agreement.
The systematic review was conducted in full compliance with the PRISMA 2020 guidelines and was formally registered in the PROSPERO database under registration number CRD420261373632. Researchers Zeynab Alizada, Aida Gafarova, Rauf Nasirov, Edin Nevzati, Antiga Muradova, and Natig Ahmadov established a structured PICOTS framework to examine peer-reviewed studies published between March 1, 2021, and March 31, 2026. The investigative team searched PubMed/MEDLINE and the Web of Science Core Collection using a structured two-block Boolean search strategy combining controlled MeSH terms and free-text keywords spanning neurosurgical diseases and procedures alongside artificial intelligence and machine learning methodologies. Databases such as Embase and IEEE Xplore were omitted from the primary search, with the authors noting that IEEE Xplore predominantly indexes engineering and computer science conference proceedings lacking the human patient-level clinical endpoints required by the review criteria. Hand-searching of reference lists supplemented the database queries to capture all relevant investigations.
Methodological quality across the primary prediction studies was formally evaluated using the Prediction Model Risk of Bias Assessment Tool, known as PROBAST, which evaluates potential bias across four distinct domains: participants, predictors, outcome, and statistical analysis. Reporting quality was assessed using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis checklist, commonly designated as TRIPOD, alongside supplementary evaluation criteria adapted for benchmark evaluations of large language models. The systematic screening protocol yielded 29 eligible publications meeting all predefined inclusion criteria. This corpus comprised 23 primary predictive model studies, two benchmark evaluations analyzing large language models on standardized neurosurgical examinations, and four systematic reviews or dedicated methodological quality appraisals. The temporal trajectory of publications showed marked expansion over the five-year period, rising from two papers in 2021 and five in 2022 to a peak of 10 in 2023, followed by six in 2024, five in 2025, and one in early 2026.
Among the 23 primary predictive model studies, computational investigations were distributed across six clinical subspecialties. Neuro-oncology and cerebrovascular neurosurgery represented the co-dominant areas of investigation, each accounting for six publications, or 26.1 percent of the primary evidence base. Spine surgery represented four studies, comprising 17.4 percent, while traumatic brain injury accounted for three studies, or 13.0 percent. The remaining four investigations were split equally between epilepsy or functional neurosurgery and pediatric neurosurgery focusing on hydrocephalus, each representing two studies, or 8.7 percent. General supervised machine learning algorithms constituted the dominant computational category across the primary literature, appearing in 16 of the 23 studies, or 69.6 percent. Combined machine learning and radiomics pipelines accounted for two studies, end-to-end deep learning architectures appeared in two studies, while hybrid deep learning and machine learning, explainable machine learning utilizing SHapley Additive exPlanations, and algorithmic complexity comparisons each represented single studies.
Across all 23 primary studies reporting numerical discrimination metrics, model performance appeared ostensibly robust. Every primary study reported an extractable area under the receiver operating characteristic curve or concordance statistic, yielding a range of 0.672 to 0.940, with a median of 0.827, a mean of 0.818, and a standard deviation of 0.092. The highest reported discrimination value in the review occurred in a spine surgery study by Santipas and colleagues evaluating cervical spine metastasis, which reported an area under the curve of 0.940 using gradient boosting; however, this estimate was derived from a small cohort of only 72 patients, severely limiting its statistical generalizability. Similarly, in neurovascular applications, Turhon and colleagues reported an internal area under the curve of 0.929 using deep learning on multi-omics computed tomography angiography data from 1,809 aneurysms across two institutions, which moderated to external discrimination values between 0.823 and 0.876.
Despite these favorable discrimination metrics, formal risk of bias evaluations using PROBAST revealed profound methodological vulnerabilities across the primary predictive literature. Only one study among the 23 primary reports, representing just 4.3 percent, was classified as having a low risk of bias. Nine studies, or 39.1 percent, demonstrated low-to-moderate risk, while 10 studies, representing 43.5 percent, exhibited high risk of bias, and three studies, or 13.0 percent, were categorized as having unclear risk. High or unclear risk classifications were predominantly driven by absent external validation, complete omission of calibration assessments, small overall sample sizes with inadequate event counts, and incomplete reporting of missing data handling. The solitary primary study achieving a low risk of bias rating was conducted by Fujimura and colleagues, published in JAMA Network Open, which developed and formally calibrated a Light Gradient Boosting Machine model to predict two-year unruptured intracranial aneurysm rupture risk across four medical centers located in Japan, the Netherlands, and the United States, outperforming established clinical scores including PHASES and UCAS.
The review identified a critical deficit in statistical calibration as one of the most pressing patient safety concerns facing neurosurgical artificial intelligence. Calibration measures the concordance between predicted probabilities generated by a mathematical model and the actual observed frequencies of clinical events. While a model can achieve a high discriminative area under the curve by correctly ranking patients relative to one another, poor calibration means it can systematically assign inaccurate absolute risk percentages to individual patients, such as assigning a 70 percent risk to individuals with a true event likelihood of 20 percent. Despite its clinical importance, formal calibration was documented in only six of the 23 primary studies, representing an overall reporting rate of just 26.1 percent. Among these six investigations, Liu and colleagues and Toh and colleagues additionally performed decision curve analysis to assess net clinical benefit, whereas Shah and colleagues reported calibration utilizing Brier scores without supplementary decision curves.
External validation practices demonstrated substantial variability across the reviewed literature. Although external testing was documented in 12 of the 23 primary studies, representing 52.2 percent and exceeding the 15 percent benchmark documented in an earlier quality audit by Warman and colleagues, the methodological rigor of these validation designs differed considerably. Certain studies utilized geographically independent multi-center cohorts across multiple continents, whereas others relied on temporal validation within single national registries. The severe consequences of validation testing were exemplified in a traumatic brain injury study by Pease and colleagues, which fused head computed tomography deep learning with clinical variables. Although the deep learning model outperformed the established International Mission for Prognosis and Analysis of Clinical Trials in TBI calculator internally with an area under the curve of 0.88 versus 0.82, external validation on the multi-center TRACK-TBI database revealed a dramatic 0.20-unit drop in discriminative performance to 0.68, significantly underperforming the standard clinical calculator.
Equally critical to the evaluation of computational modeling was the documentation of three rigorous null-result studies, which demonstrated that increasing algorithmic complexity cannot overcome the fundamental constraints of available clinical data. In pediatric epilepsy surgery, Eriksson and colleagues conducted a multi-center study at the UCL Great Ormond Street Institute testing whether advanced machine learning algorithms, larger cohorts, or automated feature selection could improve the prediction of postoperative seizure freedom; the investigation yielded a definitive null result, showing no performance improvement. Similarly, in pediatric hydrocephalus, Adil and colleagues evaluated six machine learning algorithms across more than 2,000 patients in the MarketScan database to predict endoscopic third ventriculostomy success, finding that complex models such as XGBoost and random forest failed to improve upon the traditional logistic regression Endoscopic Third Ventriculostomy Success Score, maintaining an area under the curve near 0.68. Malhotra and colleagues confirmed this finding across 761 patients from the Hydrocephalus Clinical Research Network, demonstrating that predictive capability remains fundamentally constrained by underlying biological signal quality rather than statistical complexity.
The review also examined two 2023 benchmark investigations assessing large language models on standardized neurosurgical examinations. In a study by Guerra and colleagues, GPT-4 answered 591 board questions from the Self-Assessment Neurosurgery Examination, achieving 76.6 percent accuracy and outperforming medical students and resident averages. Ali and colleagues evaluated GPT-4, GPT-3.5, and Google Bard across 149 oral board-style clinical management questions, finding that GPT-4 attained 82.6 percent accuracy with a hallucination rate of 2.3 percent on neuroimaging questions, compared to 57.1 percent for GPT-3.5. However, the authors emphasized that these examinations represent static knowledge benchmarks rather than clinical decision tools, involving no human patient-level predictive validations. Moving forward, the systematic review concludes that responsible clinical implementation will necessitate mandatory calibration reporting, prospective multi-center validation protocols, and formal impact trials to verify that algorithmic predictions translate into measurable improvements in patient outcomes.
Subject of Research: Methodological quality and predictive performance of artificial intelligence models in neurosurgery
Article Title: Predictive models in neurosurgery with machine learning, deep learning, radiomics, and large language models
Article References: Alizada, Z., Gafarova, A., Nasirov, R., Nevzati, E., Muradova, A., & Ahmadov, N. (2026). Predictive models in neurosurgery with machine learning, deep learning, radiomics, and large language models. Egyptian Journal of Neurosurgery, 41(1), Article 203. https://doi.org/10.1186/s41984-026-00698-5
Image Credits: AI Generated
DOI: 10.1186/s41984-026-00698-5
Keywords: neurosurgery, artificial intelligence, machine learning, deep learning, radiomics, large language models, systematic review, PRISMA, PROBAST, TRIPOD, model calibration, external validation, neuro-oncology, cerebrovascular neurosurgery, traumatic brain injury, spine surgery, epilepsy surgery, pediatric hydrocephalus




