Hybrid Vision Transformer for Classification of Pancreatic Cystic Lesions on Confocal Laser Endomicroscopy Videos

Clara Lavita Angelina, Yi Kai Chou, Tsung Chun Lee, Pradermchai Kongkam, Ming Lun Han, Hsiu Po Wang, Hsuan Ting Chang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The early detection of pancreatic cystic lesions plays a significant role in the survival chance of a patient with pancreatic cancer. Yet it is still a huge challenge. Unfortunately, most pancreatic cancers were diagnosed when the tumor was metastatic. In this study, the Hybrid Transformer, which is the combination of the VGG19 network and Vision Transformer, is utilized as a learning model to predict the pancreatic cystic symptom types in needle-based confocal laser endomicroscopy. A total of 16,944 images containing five types of pancreatic cystic are collected as the training and validation data. Our method can automatically classify the feature type of pancreatic cystic in the test videos and record the prediction results frame by frame. In our experiment, the proposed method successfully identifies the symptom types of 13 from 18 test videos and achieves an accuracy as high as 72%.

Original languageEnglish
Title of host publication2023 International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2023 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages47-48
Number of pages2
ISBN (Electronic)9798350324174
DOIs
Publication statusPublished - 2023
Event2023 International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2023 - Pingtung, Taiwan
Duration: Jul 17 2023Jul 19 2023

Publication series

Name2023 International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2023 - Proceedings

Conference

Conference2023 International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2023
Country/TerritoryTaiwan
CityPingtung
Period7/17/237/19/23

Keywords

  • deep learning
  • needle-based confocal laser endomicroscopy
  • pancreatic cystic symptom
  • VGG19
  • vision transformer

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction
  • Information Systems
  • Information Systems and Management
  • Electrical and Electronic Engineering
  • Media Technology
  • Instrumentation

Fingerprint

Dive into the research topics of 'Hybrid Vision Transformer for Classification of Pancreatic Cystic Lesions on Confocal Laser Endomicroscopy Videos'. Together they form a unique fingerprint.

Cite this