Go to main content

A community benchmark for the automated segmentation of pediatric neuroblastoma on multi-modal MRI: : design and results of the SPPIN challenge at MICCAI 2023

Buser, Myrthe A. D.; Simons, Dominique C.; Fitski, Matthijs; Wijnen, Marc H. W. A.; Littooij, Annemieke S.; Brugge, Annemiek H. ter; Vos, Iris N.; Janse, Markus H. A.; de Boer, Mathijs; ter Maat, Rens; Sato, Junya; Kido, Shoji; Kondo, Satoshi; Kasai, Satoshi; Wodzinski, Marek; Müller, Henning; Ye, Jin; He, Junjun; Kirchhoff, Yannick; Rokkus, Maximilian R.; Haokai, Gao; Fernández-Patón, Matías; Veiga-Canuto, Diana; Ellis, David G.; Aizenberg, Michele; van der Velden, Bas H. M.; Kuijf, Hugo; de Luca, Alberto; van der Steeg, Alida F. W.
2025
Télécharger

Surgery plays a key role in treating neuroblastoma. To assist surgical planning, anatomical 3D models derived from the segmentation of anatomical structures on MRI scans are often used. Automation using deep learning can make segmentations less time-consuming and more reliable. We organized the Surgical Planning in PedIatric Neuroblastoma (SPPIN) challenge, to stimulate developments and benchmarking of automatic segmentation of neuroblastoma on MRI. SPPIN is the first segmentation challenge in extracranial pediatric oncology. Nine teams provided a valid submission. Evaluation was based on the Dice similarity coefficient (Dice score), the 95th percentile of the Hausdorff distance (HD95), and the volumetric similarity (VS). A combination of these scores determined the ranking of the teams. The spread in the median evaluation scores per team was large (Dice: 0.21–0.82; HD95: 63.31–7.69; VS: 0.31–0.91). The top-performing team achieved a median Dice score of 0.82 (with an HD95 of 7.69 mm and a VS of 0.91) using a large, pre-trained model. However, in the pre-operative segmentations, significantly lower evaluation scores were observed. Our results indicate that pre-training might be useful in small, pediatric datasets. Although the general results of the winning team were high, they were insufficient to use for surgical planning in small, pre-operative tumors.

Titre
A community benchmark for the automated segmentation of pediatric neuroblastoma on multi-modal MRI: : design and results of the SPPIN challenge at MICCAI 2023
Auteur(s)/ trice(s)
Buser, Myrthe A. D. (Princess Máxima Center for Pediatric Oncology, 3584 CS Utrecht, The Netherlands)
Simons, Dominique C. (Princess Máxima Center for Pediatric Oncology, 3584 CS Utrecht, The Netherlands)
Fitski, Matthijs (Princess Máxima Center for Pediatric Oncology, 3584 CS Utrecht, The Netherlands)
Wijnen, Marc H. W. A. (Princess Máxima Center for Pediatric Oncology, 3584 CS Utrecht, The Netherlands)
Littooij, Annemieke S. (Princess Máxima Center for Pediatric Oncology, 3584 CS Utrecht, The Netherlands)
Brugge, Annemiek H. ter (Princess Máxima Center for Pediatric Oncology, 3584 CS Utrecht, The Netherlands)
Vos, Iris N. (Image Sciences Institute, University Medical Center Utrecht, Utrecht University, 3508 GA Utrecht, The Netherlands)
Janse, Markus H. A. (Image Sciences Institute, University Medical Center Utrecht, Utrecht University, 3508 GA Utrecht, The Netherlands)
de Boer, Mathijs (Image Sciences Institute, University Medical Center Utrecht, Utrecht University, 3508 GA Utrecht, The Netherlands)
ter Maat, Rens (Image Sciences Institute, University Medical Center Utrecht, Utrecht University, 3508 GA Utrecht, The Netherlands)
Sato, Junya (Department of Artificial Intelligence in Diagnostic Radiology, Osaka University Graduate School of Medicine, Osaka 545-8585, Japan)
Kido, Shoji (Department of Artificial Intelligence in Diagnostic Radiology, Osaka University Graduate School of Medicine, Osaka 545-8585, Japan)
Kondo, Satoshi (Department of Sciences and Informatics, Muroran Institute of Technology, Hokkaido 050-8585, Japan)
Kasai, Satoshi (Department of Intelligent Information Engineering, Fujita Health University, Aichi 470-1192, Japan)
Wodzinski, Marek (School of Management, HES-SO University of Applied Sciences and Arts Western Switzerland Valais;Department of Measurement and Electronics, University of Krakow, 30-059 Krakow, Poland)
Müller, Henning (School of Management, HES-SO University of Applied Sciences and Arts Western Switzerland Valais;Department of Radiology and Informatics, University of Geneva, 1211 Geneva, Switzerland)
Ye, Jin (Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China)
He, Junjun (Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China)
Kirchhoff, Yannick (Cancer Research Center, 69120 Heidelberg, Germany)
Rokkus, Maximilian R. (Cancer Research Center, 69120 Heidelberg, Germany)
Haokai, Gao (School of Computer Science, South China Normal University, Guangzhou 510631, China)
Fernández-Patón, Matías (La Fe Health Research Institute, 46026 Valencia, Spain)
Veiga-Canuto, Diana (La Fe Health Research Institute, 46026 Valencia, Spain)
Ellis, David G. (University of Nebraska Medical Center, Omaha, NE 68198, USA)
Aizenberg, Michele (University of Nebraska Medical Center, Omaha, NE 68198, USA)
van der Velden, Bas H. M. (Image Sciences Institute, University Medical Center Utrecht, Utrecht University, 3508 GA Utrecht, The Netherlands)
Kuijf, Hugo (Image Sciences Institute, University Medical Center Utrecht, Utrecht University, 3508 GA Utrecht, The Netherlands)
de Luca, Alberto (Image Sciences Institute, University Medical Center Utrecht, Utrecht University, 3508 GA Utrecht, The Netherlands)
van der Steeg, Alida F. W. (Princess Máxima Center for Pediatric Oncology, 3584 CS Utrecht, The Netherlands)
Date
2025-10
Publié dans
Bioengineering
Volume
2025, 12
Numéro
11
Publié par
New York, MDPI
Pagination & équivalents
16 p.
Présenté à
MICCAI 2023, Vancouver, 2023-10-08, 2023-10-12
DOI
https://doi.org/10.3390/bioengineering12111157
Type de papier
published full paper
Domaine
Economie et Services
Ecole
HEG-VS
Institut
Institut Informatique de gestion
Le document apparaît dans
Documents de conférences
Global