LBERT: Lexically aware Transformer-based Bidirectional Encoder Representation model for learning universal bio-entity relations

Neha Warikoo, Yung Chun Chang, Wen Lian Hsu

研究成果: 雜誌貢獻文章同行評審

7 引文 斯高帕斯(Scopus)


MOTIVATION: Natural Language Processing techniques are constantly being advanced to accommodate the influx of data as well as to provide exhaustive and structured knowledge dissemination. Within the biomedical domain, relation detection between bio-entities known as the Bio-Entity Relation Extraction (BRE) task has a critical function in knowledge structuring. Although recent advances in deep learning-based biomedical domain embedding have improved BRE predictive analytics, these works are often task selective or use external knowledge-based pre-/post-processing. In addition, deep learning-based models do not account for local syntactic contexts, which have improved data representation in many kernel classifier-based models. In this study, we propose a universal BRE model, i.e. LBERT, which is a Lexically aware Transformer-based Bidirectional Encoder Representation model, and which explores both local and global contexts representations for sentence-level classification tasks. RESULTS: This article presents one of the most exhaustive BRE studies ever conducted over five different bio-entity relation types. Our model outperforms state-of-the-art deep learning models in protein-protein interaction (PPI), drug-drug interaction and protein-bio-entity relation classification tasks by 0.02%, 11.2% and 41.4%, respectively. LBERT representations show a statistically significant improvement over BioBERT in detecting true bio-entity relation for large corpora like PPI. Our ablation studies clearly indicate the contribution of the lexical features and distance-adjusted attention in improving prediction performance by learning additional local semantic context along with bi-directionally learned global context. AVAILABILITY AND IMPLEMENTATION: Github. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

頁(從 - 到)404-412
期刊Bioinformatics (Oxford, England)
出版狀態已發佈 - 4月 20 2021

ASJC Scopus subject areas

  • 計算數學
  • 分子生物學
  • 生物化學
  • 統計與概率
  • 電腦科學應用
  • 計算機理論與數學


深入研究「LBERT: Lexically aware Transformer-based Bidirectional Encoder Representation model for learning universal bio-entity relations」主題。共同形成了獨特的指紋。