AMP-BiLSTM: An Enhanced Highlight Extraction Method Using Multi-Channel Bi-LSTM and Self-Attention in Streaming Videos

Sheng Jie Lin, Chien Chin Chen, Yung Chun Chang

研究成果: 書貢獻/報告類型會議貢獻

摘要

With the rise of conversation-oriented streaming videos, the platforms that host them like Twitch have rapidly become prominent information hubs. However, the lengthy nature of such streams often deters viewers from consuming the full content. To mitigate this, we propose AMP-BiLSTM, a novel highlight extraction method which focuses on the textual information in streamer discourses and viewer responses rather than visual features. This approach addresses the limitations of previous methods, which primarily centered on analyzing visual features, and were thus insufficient for conversation-oriented streams where highlights emerge from dialogues and viewer interactions. AMP-BiLSTM is built on techniques of Attention, Multi-channel, and Position enrichment integrated into a Bidirectional Long Short-Term Memory (BiLSTM) network. Through experiments on a real-world dataset, we found that streamer discourses and viewer messages provide significant utility for highlight extraction in conversation-oriented streaming videos. Furthermore, our proposed Multi-channel and self-attention techniques effectively distill text streams into semantically-rich embeddings. The experiment results demonstrate that AMP-BiLSTM outperforms several state-of-the-art methods for deep learning-based highlight extraction, thus showing promise for improved conversation-oriented streaming video content digestion.
原文英語
主出版物標題Proceedings - 18th IEEE International Conference on Semantic Computing, ICSC 2024
發行者Institute of Electrical and Electronics Engineers Inc.
頁面9-16
頁數8
ISBN(電子)9798350385359
DOIs
出版狀態已發佈 - 2024
事件18th IEEE International Conference on Semantic Computing, ICSC 2024 - Hybrid, Laguna Hills, 美国
持續時間: 2月 5 20242月 7 2024

出版系列

名字Proceedings - IEEE International Conference on Semantic Computing, ICSC
ISSN(列印)2325-6516
ISSN(電子)2472-9671

會議

會議18th IEEE International Conference on Semantic Computing, ICSC 2024
國家/地區美国
城市Hybrid, Laguna Hills
期間2/5/242/7/24

ASJC Scopus subject areas

  • 人工智慧
  • 電腦網路與通信
  • 人機介面
  • 資訊系統與管理

指紋

深入研究「AMP-BiLSTM: An Enhanced Highlight Extraction Method Using Multi-Channel Bi-LSTM and Self-Attention in Streaming Videos」主題。共同形成了獨特的指紋。

引用此