Valmet: A new validation tool for assessing and improving 3D object segmentation

Guido Gerig, Matthieu Jomier, Miranda Chakos

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Extracting 3D structures from volumetric images like MRI or CT is becoming a routine process for diagnosis based on quantitation, for radiotherapy planning, for surgical planning and image-guided intervention, for studying neurodevelopmental and neurodegenerative aspects of brain diseases, and for clinical drug trials. Key issues for segmenting anatomical objects from 3D medical images are validity and reliability. We have developed VALMET, a new tool for validation and comparison of object segmentation. New features not available in commercial and public-domain image processing packages are the choice between different metrics to describe differences between segmentations and the use of graphical overlay and 3D display for visual assessment of the locality and magnitude of segmentation variability. Input to the tool are an original 3D image (MRI, CT, ultrasound), and a series of segmentations either generated by several human raters and/or by automatic methods (machine). Quantitative evaluation includes intra-class correlation of resulting volumes and four different shape distance metrics, a) percentage overlap of segmented structures (R intersect S)/(R union S), b) probabilistic overlap measure for non-binary segmentations, c) mean/median absolute distances between object surfaces, and maximum (Hausdorff) distance. All these measures are calculated for arbitrarily selected 2D cross-sections and full 3D segmentations. Segmentation results are overlaid onto the original image data for visual comparison. A 3D graphical display of the segmented organ is color-coded depending on the selected metric for measuring segmentation difference. The new tool is in routine use for intra- and inter-rater reliability studies and for testing novel automatic machine-segmentation versus a gold standard established by human experts. Preliminary studies showed that the new tool could significantly improve intra- and inter-rater reliability of hippocampus segmentation to achieve intra-class correlation coefficients significantly higher than published elsewhere.

    Original languageEnglish (US)
    Title of host publicationMedical Image Computing and Computer-Assisted Intervention - MICCAI 2001 - 4th International Conference, Proceedings
    PublisherSpringer Verlag
    Pages516-523
    Number of pages8
    Volume2208
    ISBN (Print)3540426973, 9783540454687
    DOIs
    StatePublished - 2001
    Event4th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2001 - Utrecht, Netherlands
    Duration: Oct 14 2001Oct 17 2001

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume2208
    ISSN (Print)03029743
    ISSN (Electronic)16113349

    Other

    Other4th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2001
    CountryNetherlands
    CityUtrecht
    Period10/14/0110/17/01

    Fingerprint

    Segmentation
    Magnetic resonance imaging
    Display devices
    Automatic testing
    Planning
    Radiotherapy
    3D Display
    3D Image
    Brain
    Image processing
    Ultrasonics
    Overlap
    Color
    Object
    Intraclass Correlation Coefficient
    Intraclass Correlation
    Graphical Display
    Hippocampus
    Metric
    Hausdorff Distance

    ASJC Scopus subject areas

    • Computer Science(all)
    • Theoretical Computer Science

    Cite this

    Gerig, G., Jomier, M., & Chakos, M. (2001). Valmet: A new validation tool for assessing and improving 3D object segmentation. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2001 - 4th International Conference, Proceedings (Vol. 2208, pp. 516-523). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2208). Springer Verlag. https://doi.org/10.1007/3-540-45468-3_62

    Valmet : A new validation tool for assessing and improving 3D object segmentation. / Gerig, Guido; Jomier, Matthieu; Chakos, Miranda.

    Medical Image Computing and Computer-Assisted Intervention - MICCAI 2001 - 4th International Conference, Proceedings. Vol. 2208 Springer Verlag, 2001. p. 516-523 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2208).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Gerig, G, Jomier, M & Chakos, M 2001, Valmet: A new validation tool for assessing and improving 3D object segmentation. in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2001 - 4th International Conference, Proceedings. vol. 2208, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 2208, Springer Verlag, pp. 516-523, 4th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2001, Utrecht, Netherlands, 10/14/01. https://doi.org/10.1007/3-540-45468-3_62
    Gerig G, Jomier M, Chakos M. Valmet: A new validation tool for assessing and improving 3D object segmentation. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2001 - 4th International Conference, Proceedings. Vol. 2208. Springer Verlag. 2001. p. 516-523. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/3-540-45468-3_62
    Gerig, Guido ; Jomier, Matthieu ; Chakos, Miranda. / Valmet : A new validation tool for assessing and improving 3D object segmentation. Medical Image Computing and Computer-Assisted Intervention - MICCAI 2001 - 4th International Conference, Proceedings. Vol. 2208 Springer Verlag, 2001. pp. 516-523 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{bbff4104b6124d33963160b09c2b53c8,
    title = "Valmet: A new validation tool for assessing and improving 3D object segmentation",
    abstract = "Extracting 3D structures from volumetric images like MRI or CT is becoming a routine process for diagnosis based on quantitation, for radiotherapy planning, for surgical planning and image-guided intervention, for studying neurodevelopmental and neurodegenerative aspects of brain diseases, and for clinical drug trials. Key issues for segmenting anatomical objects from 3D medical images are validity and reliability. We have developed VALMET, a new tool for validation and comparison of object segmentation. New features not available in commercial and public-domain image processing packages are the choice between different metrics to describe differences between segmentations and the use of graphical overlay and 3D display for visual assessment of the locality and magnitude of segmentation variability. Input to the tool are an original 3D image (MRI, CT, ultrasound), and a series of segmentations either generated by several human raters and/or by automatic methods (machine). Quantitative evaluation includes intra-class correlation of resulting volumes and four different shape distance metrics, a) percentage overlap of segmented structures (R intersect S)/(R union S), b) probabilistic overlap measure for non-binary segmentations, c) mean/median absolute distances between object surfaces, and maximum (Hausdorff) distance. All these measures are calculated for arbitrarily selected 2D cross-sections and full 3D segmentations. Segmentation results are overlaid onto the original image data for visual comparison. A 3D graphical display of the segmented organ is color-coded depending on the selected metric for measuring segmentation difference. The new tool is in routine use for intra- and inter-rater reliability studies and for testing novel automatic machine-segmentation versus a gold standard established by human experts. Preliminary studies showed that the new tool could significantly improve intra- and inter-rater reliability of hippocampus segmentation to achieve intra-class correlation coefficients significantly higher than published elsewhere.",
    author = "Guido Gerig and Matthieu Jomier and Miranda Chakos",
    year = "2001",
    doi = "10.1007/3-540-45468-3_62",
    language = "English (US)",
    isbn = "3540426973",
    volume = "2208",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    publisher = "Springer Verlag",
    pages = "516--523",
    booktitle = "Medical Image Computing and Computer-Assisted Intervention - MICCAI 2001 - 4th International Conference, Proceedings",

    }

    TY - GEN

    T1 - Valmet

    T2 - A new validation tool for assessing and improving 3D object segmentation

    AU - Gerig, Guido

    AU - Jomier, Matthieu

    AU - Chakos, Miranda

    PY - 2001

    Y1 - 2001

    N2 - Extracting 3D structures from volumetric images like MRI or CT is becoming a routine process for diagnosis based on quantitation, for radiotherapy planning, for surgical planning and image-guided intervention, for studying neurodevelopmental and neurodegenerative aspects of brain diseases, and for clinical drug trials. Key issues for segmenting anatomical objects from 3D medical images are validity and reliability. We have developed VALMET, a new tool for validation and comparison of object segmentation. New features not available in commercial and public-domain image processing packages are the choice between different metrics to describe differences between segmentations and the use of graphical overlay and 3D display for visual assessment of the locality and magnitude of segmentation variability. Input to the tool are an original 3D image (MRI, CT, ultrasound), and a series of segmentations either generated by several human raters and/or by automatic methods (machine). Quantitative evaluation includes intra-class correlation of resulting volumes and four different shape distance metrics, a) percentage overlap of segmented structures (R intersect S)/(R union S), b) probabilistic overlap measure for non-binary segmentations, c) mean/median absolute distances between object surfaces, and maximum (Hausdorff) distance. All these measures are calculated for arbitrarily selected 2D cross-sections and full 3D segmentations. Segmentation results are overlaid onto the original image data for visual comparison. A 3D graphical display of the segmented organ is color-coded depending on the selected metric for measuring segmentation difference. The new tool is in routine use for intra- and inter-rater reliability studies and for testing novel automatic machine-segmentation versus a gold standard established by human experts. Preliminary studies showed that the new tool could significantly improve intra- and inter-rater reliability of hippocampus segmentation to achieve intra-class correlation coefficients significantly higher than published elsewhere.

    AB - Extracting 3D structures from volumetric images like MRI or CT is becoming a routine process for diagnosis based on quantitation, for radiotherapy planning, for surgical planning and image-guided intervention, for studying neurodevelopmental and neurodegenerative aspects of brain diseases, and for clinical drug trials. Key issues for segmenting anatomical objects from 3D medical images are validity and reliability. We have developed VALMET, a new tool for validation and comparison of object segmentation. New features not available in commercial and public-domain image processing packages are the choice between different metrics to describe differences between segmentations and the use of graphical overlay and 3D display for visual assessment of the locality and magnitude of segmentation variability. Input to the tool are an original 3D image (MRI, CT, ultrasound), and a series of segmentations either generated by several human raters and/or by automatic methods (machine). Quantitative evaluation includes intra-class correlation of resulting volumes and four different shape distance metrics, a) percentage overlap of segmented structures (R intersect S)/(R union S), b) probabilistic overlap measure for non-binary segmentations, c) mean/median absolute distances between object surfaces, and maximum (Hausdorff) distance. All these measures are calculated for arbitrarily selected 2D cross-sections and full 3D segmentations. Segmentation results are overlaid onto the original image data for visual comparison. A 3D graphical display of the segmented organ is color-coded depending on the selected metric for measuring segmentation difference. The new tool is in routine use for intra- and inter-rater reliability studies and for testing novel automatic machine-segmentation versus a gold standard established by human experts. Preliminary studies showed that the new tool could significantly improve intra- and inter-rater reliability of hippocampus segmentation to achieve intra-class correlation coefficients significantly higher than published elsewhere.

    UR - http://www.scopus.com/inward/record.url?scp=84958182494&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84958182494&partnerID=8YFLogxK

    U2 - 10.1007/3-540-45468-3_62

    DO - 10.1007/3-540-45468-3_62

    M3 - Conference contribution

    AN - SCOPUS:84958182494

    SN - 3540426973

    SN - 9783540454687

    VL - 2208

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 516

    EP - 523

    BT - Medical Image Computing and Computer-Assisted Intervention - MICCAI 2001 - 4th International Conference, Proceedings

    PB - Springer Verlag

    ER -