Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information

Qiangguo Jin; Changjiang Zou; Hui Cui; Changming Sun; Shu Wei Huang; Yi Jie Kuo; Ping Xuan; Leilei Cao; Ran Su; Leyi Wei; Henry B.L. Duh; Yu Pin Chen

doi:10.1007/978-3-031-43987-2_9

Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information

Qiangguo Jin, Changjiang Zou, Hui Cui, Changming Sun, Shu Wei Huang, Yi Jie Kuo, Ping Xuan, Leilei Cao, Ran Su, Leyi Wei, Henry B.L. Duh, Yu Pin Chen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Citations (Scopus)

Abstract

Sarcopenia is a condition of age-associated muscle degeneration that shortens the life expectancy in those it affects, compared to individuals with normal muscle strength. Accurate screening for sarcopenia is a key process of clinical diagnosis and therapy. In this work, we propose a novel multi-modality contrastive learning (MM-CL) based method that combines hip X-ray images and clinical parameters for sarcopenia screening. Our method captures the long-range information with Non-local CAM Enhancement, explores the correlations in visual-text features via Visual-text Feature Fusion, and improves the model’s feature representation ability through Auxiliary contrastive representation. Furthermore, we establish a large in-house dataset with 1,176 patients to validate the effectiveness of multi-modality based methods. Significant performances with an AUC of 84.64%, ACC of 79.93%, F1 of 74.88%, SEN of 72.06%, SPC of 86.06%, and PRE of 78.44%, show that our method outperforms other single-modality and multi-modality based methods.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings
Editors	Hayit Greenspan, Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	85-94
Number of pages	10
ISBN (Print)	9783031439865
DOIs	https://doi.org/10.1007/978-3-031-43987-2_9
Publication status	Published - 2023
Event	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 - Vancouver, Canada Duration: Oct 8 2023 → Oct 12 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14225 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
Country/Territory	Canada
City	Vancouver
Period	10/8/23 → 10/12/23

Keywords

Contrastive learning
Multi-modality feature fusion
Sarcopenia screening

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/978-3-031-43987-2_9

Cite this

Jin, Q., Zou, C., Cui, H., Sun, C., Huang, S. W., Kuo, Y. J., Xuan, P., Cao, L., Su, R., Wei, L., Duh, H. B. L., & Chen, Y. P. (2023). Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. In H. Greenspan, H. Greenspan, A. Madabhushi, P. Mousavi, S. Salcudean, J. Duncan, T. Syeda-Mahmood, & R. Taylor (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings (pp. 85-94). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14225 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-43987-2_9

Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. / Jin, Qiangguo; Zou, Changjiang; Cui, Hui et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. ed. / Hayit Greenspan; Hayit Greenspan; Anant Madabhushi; Parvin Mousavi; Septimiu Salcudean; James Duncan; Tanveer Syeda-Mahmood; Russell Taylor. Springer Science and Business Media Deutschland GmbH, 2023. p. 85-94 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14225 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Jin, Q, Zou, C, Cui, H, Sun, C, Huang, SW, Kuo, YJ, Xuan, P, Cao, L, Su, R, Wei, L, Duh, HBL & Chen, YP 2023, Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. in H Greenspan, H Greenspan, A Madabhushi, P Mousavi, S Salcudean, J Duncan, T Syeda-Mahmood & R Taylor (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14225 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 85-94, 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023, Vancouver, Canada, 10/8/23. https://doi.org/10.1007/978-3-031-43987-2_9

Jin Q, Zou C, Cui H, Sun C, Huang SW, Kuo YJ et al. Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. In Greenspan H, Greenspan H, Madabhushi A, Mousavi P, Salcudean S, Duncan J, Syeda-Mahmood T, Taylor R, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 85-94. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-43987-2_9

Jin, Qiangguo ; Zou, Changjiang ; Cui, Hui et al. / Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. editor / Hayit Greenspan ; Hayit Greenspan ; Anant Madabhushi ; Parvin Mousavi ; Septimiu Salcudean ; James Duncan ; Tanveer Syeda-Mahmood ; Russell Taylor. Springer Science and Business Media Deutschland GmbH, 2023. pp. 85-94 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{957fb77206354dfd8b1800a335469bce,

title = "Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information",

abstract = "Sarcopenia is a condition of age-associated muscle degeneration that shortens the life expectancy in those it affects, compared to individuals with normal muscle strength. Accurate screening for sarcopenia is a key process of clinical diagnosis and therapy. In this work, we propose a novel multi-modality contrastive learning (MM-CL) based method that combines hip X-ray images and clinical parameters for sarcopenia screening. Our method captures the long-range information with Non-local CAM Enhancement, explores the correlations in visual-text features via Visual-text Feature Fusion, and improves the model{\textquoteright}s feature representation ability through Auxiliary contrastive representation. Furthermore, we establish a large in-house dataset with 1,176 patients to validate the effectiveness of multi-modality based methods. Significant performances with an AUC of 84.64%, ACC of 79.93%, F1 of 74.88%, SEN of 72.06%, SPC of 86.06%, and PRE of 78.44%, show that our method outperforms other single-modality and multi-modality based methods.",

keywords = "Contrastive learning, Multi-modality feature fusion, Sarcopenia screening",

author = "Qiangguo Jin and Changjiang Zou and Hui Cui and Changming Sun and Huang, {Shu Wei} and Kuo, {Yi Jie} and Ping Xuan and Leilei Cao and Ran Su and Leyi Wei and Duh, {Henry B.L.} and Chen, {Yu Pin}",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.; 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 ; Conference date: 08-10-2023 Through 12-10-2023",

year = "2023",

doi = "10.1007/978-3-031-43987-2_9",

language = "English",

isbn = "9783031439865",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "85--94",

editor = "Hayit Greenspan and Hayit Greenspan and Anant Madabhushi and Parvin Mousavi and Septimiu Salcudean and James Duncan and Tanveer Syeda-Mahmood and Russell Taylor",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings",

address = "Germany",

}

TY - GEN

T1 - Multi-modality Contrastive Learning for Sarcopenia Screening from Hip X-rays and Clinical Information

AU - Jin, Qiangguo

AU - Zou, Changjiang

AU - Cui, Hui

AU - Sun, Changming

AU - Huang, Shu Wei

AU - Kuo, Yi Jie

AU - Xuan, Ping

AU - Cao, Leilei

AU - Su, Ran

AU - Wei, Leyi

AU - Duh, Henry B.L.

AU - Chen, Yu Pin

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.

PY - 2023

Y1 - 2023

N2 - Sarcopenia is a condition of age-associated muscle degeneration that shortens the life expectancy in those it affects, compared to individuals with normal muscle strength. Accurate screening for sarcopenia is a key process of clinical diagnosis and therapy. In this work, we propose a novel multi-modality contrastive learning (MM-CL) based method that combines hip X-ray images and clinical parameters for sarcopenia screening. Our method captures the long-range information with Non-local CAM Enhancement, explores the correlations in visual-text features via Visual-text Feature Fusion, and improves the model’s feature representation ability through Auxiliary contrastive representation. Furthermore, we establish a large in-house dataset with 1,176 patients to validate the effectiveness of multi-modality based methods. Significant performances with an AUC of 84.64%, ACC of 79.93%, F1 of 74.88%, SEN of 72.06%, SPC of 86.06%, and PRE of 78.44%, show that our method outperforms other single-modality and multi-modality based methods.

AB - Sarcopenia is a condition of age-associated muscle degeneration that shortens the life expectancy in those it affects, compared to individuals with normal muscle strength. Accurate screening for sarcopenia is a key process of clinical diagnosis and therapy. In this work, we propose a novel multi-modality contrastive learning (MM-CL) based method that combines hip X-ray images and clinical parameters for sarcopenia screening. Our method captures the long-range information with Non-local CAM Enhancement, explores the correlations in visual-text features via Visual-text Feature Fusion, and improves the model’s feature representation ability through Auxiliary contrastive representation. Furthermore, we establish a large in-house dataset with 1,176 patients to validate the effectiveness of multi-modality based methods. Significant performances with an AUC of 84.64%, ACC of 79.93%, F1 of 74.88%, SEN of 72.06%, SPC of 86.06%, and PRE of 78.44%, show that our method outperforms other single-modality and multi-modality based methods.

KW - Contrastive learning

KW - Multi-modality feature fusion

KW - Sarcopenia screening

UR - http://www.scopus.com/inward/record.url?scp=85174743637&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85174743637&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-43987-2_9

DO - 10.1007/978-3-031-43987-2_9

M3 - Conference contribution

AN - SCOPUS:85174743637

SN - 9783031439865

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 85

EP - 94

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings

A2 - Greenspan, Hayit

A2 - Madabhushi, Anant

A2 - Mousavi, Parvin

A2 - Salcudean, Septimiu

A2 - Duncan, James

A2 - Syeda-Mahmood, Tanveer

A2 - Taylor, Russell

PB - Springer Science and Business Media Deutschland GmbH

T2 - 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023

Y2 - 8 October 2023 through 12 October 2023

ER -