(Telkom University, Indonesia)(2) Isman Kurniawan
(Telkom University, Indonesia)*corresponding author
AbstractDrug development requires accurate drug-target interaction (DTI) information to evaluate a drug's potential. However, existing current methods for estimating DTI are slow and expensive. Deep learning offers an efficient and effective alternative by leveraging sequence data for prediction. Nevertheless, the DTI binary classification approach suffers from a large number of non-interacting pairs, resulting in data imbalance and has a negative impact on performance. To address this issue, DTI is modeled as a regression problem known as drug-target affinity (DTA), which predicts the strength of interactions. While various deep learning methods show competitive results in DTA prediction, they face a challenge in capturing specific drug-target patterns with limited data. To overcome the problem, this study leverages pre-trained language models for enhanced representation. Also, we utilize gated multi-head attention (GMHA), which modifies multi-head attention by including dynamic scaling and a gate process to capture the mutual interactions better. The results show that our proposed method exceeds the benchmark and baseline in all evaluation metrics, with concordance index (CI) of 0.893 and 0.872, and modified r-squared (rm2) of 0.673 and 0.723 in Davis and KIBA. Our findings further suggest that pre-trained language models for drug and target receptor representation improve DTA prediction model performance. Also, the GMHA method generally outperforms the simple concatenation method, with more obvious advantages in more complex datasets like KIBA. Our approach provides a competitive enhancement in DTA prediction, suggesting a promising direction for further enhancing drug discovery and development processes.
KeywordsDrug target affinity; Pre-trained language model; Gated multi-head attention; Deep learning; Regression
|
DOIhttps://doi.org/10.26555/ijain.v11i2.1910 |
Article metricsAbstract views : 2060 | PDF views : 282 |
Cite |
Full Text Download
|
References
[1] Y.-F. Zhang et al., “SPVec: A Word2vec-Inspired Feature Representation Method for Drug-Target Interaction Prediction,” Front. Chem., vol. 7, p. 504142, Jan. 2020, doi: 10.3389/fchem.2019.00895.
[2] H. Khojasteh, J. Pirgazi, and A. Ghanbari Sorkhi, “Improving prediction of drug-target interactions based on fusing multiple features with data balancing and feature selection techniques,” PLoS One, vol. 18, no. 8, p. e0288173, Aug. 2023, doi: 10.1371/journal.pone.0288173.
[3] S. Kim et al., “PubChem 2019 update: improved access to chemical data,” Nucleic Acids Res., vol. 47, no. D1, pp. D1102–D1109, Jan. 2019, doi: 10.1093/nar/gky1033.
[4] C. H. Wong, K. W. Siah, and A. W. Lo, “Estimation of clinical trial success rates and related parameters,” Biostatistics, vol. 20, no. 2, pp. 273–286, Apr. 2019, doi: 10.1093/biostatistics/kxx069.
[5] M. Kalemati, M. Zamani Emani, and S. Koohi, “BiComp-DTA: Drug-target binding affinity prediction through complementary biological-related and compression-based featurization approach,” PLOS Comput. Biol., vol. 19, no. 3, p. e1011036, Mar. 2023, doi: 10.1371/journal.pcbi.1011036.
[6] S. Lin, C. Shi, and J. Chen, “GeneralizedDTA: combining pre-training and multi-task learning to predict drug-target binding affinity for unknown drug discovery,” BMC Bioinformatics, vol. 23, no. 1, p. 367, Sep. 2022, doi: 10.1186/s12859-022-04905-6.
[7] H. Abbasi Mesrabadi, K. Faez, and J. Pirgazi, “Drug–target interaction prediction based on protein features, using wrapper feature selection,” Sci. Rep., vol. 13, no. 1, p. 3594, Mar. 2023, doi: 10.1038/s41598-023-30026-y.
[8] L. Douali, “Machine learning for the prediction of phenols cytotoxicity,” Int. J. Adv. Intell. Informatics, vol. 8, no. 1, p. 58, Mar. 2022, doi: 10.26555/ijain.v8i1.748.
[9] Y. Qian, X. Li, J. Wu, and Q. Zhang, “MCL-DTI: using drug multimodal information and bi-directional cross-attention learning method for predicting drug–target interaction,” BMC Bioinformatics, vol. 24, no. 1, p. 323, Aug. 2023, doi: 10.1186/s12859-023-05447-1.
[10] Z.-H. Ren et al., “DeepMPF: deep learning framework for predicting drug–target interactions based on multi-modal representation with meta-path semantic analysis,” J. Transl. Med., vol. 21, no. 1, p. 48, Jan. 2023, doi: 10.1186/s12967-023-03876-3.
[11] A. Saad, F. A. Maghraby, and Y. M. Omar, “Predicting Drug Target Interaction by Integrating Drug Fingerprint and Drug Side Effect Using Machine Learning,” in Advances in Intelligent Systems and Computing, vol. 921, Springer, Cham, 2020, pp. 281–290, doi: 10.1007/978-3-030-14118-9_28.
[12] A. Mahdaddi, S. Meshoul, and M. Belguidoum, “EA-based hyperparameter optimization of hybrid deep learning models for effective drug-target interactions prediction,” Expert Syst. Appl., vol. 185, p. 115525, Dec. 2021, doi: 10.1016/j.eswa.2021.115525.
[13] H. Öztürk, A. Özgür, and E. Ozkirimli, “DeepDTA: deep drug–target binding affinity prediction,” Bioinformatics, vol. 34, no. 17, pp. i821–i829, Sep. 2018, doi: 10.1093/bioinformatics/bty593.
[14] Q. Zhao, G. Duan, M. Yang, Z. Cheng, Y. Li, and J. Wang, “AttentionDTA: Drug–Target Binding Affinity Prediction by Sequence-Based Deep Learning With Attention Mechanism,” IEEE/ACM Trans. Comput. Biol. Bioinforma., vol. 20, no. 2, pp. 852–863, Mar. 2023, doi: 10.1109/TCBB.2022.3170365.
[15] T. He, M. Heidemeyer, F. Ban, A. Cherkasov, and M. Ester, “SimBoost: a read-across approach for predicting drug–target binding affinities using gradient boosting machines,” J. Cheminform., vol. 9, no. 1, p. 24, Dec. 2017, doi: 10.1186/s13321-017-0209-z.
[16] M. A. Thafar, M. Alshahrani, S. Albaradei, T. Gojobori, M. Essack, and X. Gao, “Affinity2Vec: drug-target binding affinity prediction through representation learning, graph mining, and machine learning,” Sci. Rep., vol. 12, no. 1, p. 4751, Mar. 2022, doi: 10.1038/s41598-022-08787-9.
[17] H. Öztürk, E. Ozkirimli, and A. Özgür, “WideDTA: prediction of drug-target binding affinity,” arxiv Artif. Intell., pp. 1–11, 2019, [Online]. Available at: http://arxiv.org/abs/1902.04166.
[18] A. Ghimire, H. Tayara, Z. Xuan, and K. T. Chong, “CSatDTA: Prediction of Drug–Target Binding Affinity Using Convolution Model with Self-Attention,” Int. J. Mol. Sci., vol. 23, no. 15, p. 8453, Jul. 2022, doi: 10.3390/ijms23158453.
[19] S. D’Souza, K. V. Prema, S. Balaji, and R. Shah, “Deep Learning-Based Modeling of Drug–Target Interaction Prediction Incorporating Binding Site Information of Proteins,” Interdiscip. Sci. Comput. Life Sci., vol. 15, no. 2, pp. 306–315, Jun. 2023, doi: 10.1007/s12539-023-00557-z.
[20] H. Chen, D. Li, J. Liao, L. Wei, and L. Wei, “MultiscaleDTA: A multiscale-based method with a self-attention mechanism for drug-target binding affinity prediction,” Methods, vol. 207, pp. 103–109, Nov. 2022, doi: 10.1016/j.ymeth.2022.09.006.
[21] X. Zhu, J. Liu, J. Zhang, Z. Yang, F. Yang, and X. Zhang, “FingerDTA: A Fingerprint-Embedding Framework for Drug-Target Binding Affinity Prediction,” Big Data Min. Anal., vol. 6, no. 1, pp. 1–10, Mar. 2023, doi: 10.26599/BDMA.2022.9020005.
[22] Y. Zeng, X. Chen, Y. Luo, X. Li, and D. Peng, “Deep drug-target binding affinity prediction with multiple attention blocks,” Brief. Bioinform., vol. 22, no. 5, pp. 1–10, Sep. 2021, doi: 10.1093/bib/bbab117.
[23] K. Abbasi, P. Razzaghi, A. Poso, M. Amanlou, J. B. Ghasemi, and A. Masoudi-Nejad, “DeepCDA: deep cross-domain compound–protein affinity prediction through LSTM and convolutional neural networks,” Bioinformatics, vol. 36, no. 17, pp. 4633–4642, Nov. 2020, doi: 10.1093/bioinformatics/btaa544.
[24] T. M. Nguyen, T. Nguyen, and T. Tran, “Mitigating cold-start problems in drug-target affinity prediction with interaction knowledge transferring,” Brief. Bioinform., vol. 23, no. 4, pp. 1–13, Jul. 2022, doi: 10.1093/bib/bbac269.
[25] B. Min et al., “Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey,” ACM Comput. Surv., vol. 56, no. 2, pp. 1–40, Feb. 2024, doi: 10.1145/3605943.
[26] W. Ahmad, E. Simon, S. Chithrananda, G. Grand, and B. Ramsundar, “ChemBERTa-2: Towards Chemical Foundation Models,” arxiv Artif. Intell., pp. 1–8, Sep. 2022. [Online]. Available at: https://arxiv.org/abs/2209.01712v1.
[27] Z. Lin et al., “Evolutionary-scale prediction of atomic level protein structure with a language model,” bioRxiv. Cold Spring Harbor Laboratory, p. 2022.07.20.500902, Jul. 21, 2022, doi: 10.1101/2022.07.20.500902.
[28] S. Chithrananda, G. Grand, and B. R. Deepchem, “ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction,” arxiv Artif. Intell., pp. 1–8, Oct. 2020. [Online]. Available at: https://arxiv.org/abs/2010.09885.
[29] B. Fabian et al., “Molecular representation learning with language models and domain-relevant auxiliary tasks,” arXiv, pp. 1–12, Nov. 2020. [Online]. Available at: https://arxiv.org/abs/2011.13230v1.
[30] A. Elnaggar et al., “ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Learning,” bioRxiv. Cold Spring Harbor Laboratory, p. 2020.07.12.199554, Jul. 12, 2020, doi: 10.1101/2020.07.12.199554.
[31] B. E. Suzek, H. Huang, P. McGarvey, R. Mazumder, and C. H. Wu, “UniRef: comprehensive and non-redundant UniProt reference clusters,” Bioinformatics, vol. 23, no. 10, pp. 1282–1288, May 2007, doi: 10.1093/bioinformatics/btm098.
[32] A. Ranjan, M. S. Fahad, and A. Deepak, “Scaled-attention: A novel fast attention mechanism for efficient modeling of protein sequences,” Inf. Sci. (Ny)., vol. 609, pp. 1098–1112, Sep. 2022, doi: 10.1016/j.ins.2022.07.127.
[33] K. Kurnianingsih et al., “Big data analytics for relative humidity time series forecasting based on the LSTM network and ELM,” Int. J. Adv. Intell. Informatics, vol. 9, no. 3, p. 537, Nov. 2023, doi: 10.26555/ijain.v9i3.905.
[34] J. Tang et al., “Making Sense of Large-Scale Kinase Inhibitor Bioactivity Data Sets: A Comparative and Integrative Analysis,” J. Chem. Inf. Model., vol. 54, no. 3, pp. 735–743, Mar. 2014, doi: 10.1021/ci400709d.
[35] M. I. Davis et al., “Comprehensive analysis of kinase inhibitor selectivity,” Nat. Biotechnol., vol. 29, no. 11, pp. 1046–1051, Nov. 2011, doi: 10.1038/nbt.1990.
[36] M. Lee, “Recent Advances in Deep Learning for Protein-Protein Interaction Analysis: A Comprehensive Review,” Molecules, vol. 28, no. 13, p. 5169, Jul. 2023, doi: 10.3390/molecules28135169.
[37] A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 2017-Decem, no. Nips, pp. 5999–6009, 2017, [Online]. Available at: https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
[38] H. Kang, S. Goo, H. Lee, J.-W. Chae, H.-Y. Yun, and S. Jung, “Fine-tuning of BERT Model to Accurately Predict Drug–Target Interactions,” Pharmaceutics, vol. 14, no. 8, p. 1710, Aug. 2022, doi: 10.3390/pharmaceutics14081710.
[39] T. Nguyen, H. Le, T. P. Quinn, T. Nguyen, T. D. Le, and S. Venkatesh, “GraphDTA: predicting drug–target binding affinity with graph neural networks,” Bioinformatics, vol. 37, no. 8, pp. 1140–1147, May 2021, doi: 10.1093/bioinformatics/btaa921.
[40] M. Gönen and G. Heller, “Concordance probability and discriminatory power in proportional hazards regression,” Biometrika, vol. 92, no. 4, pp. 965–970, Dec. 2005, doi: 10.1093/biomet/92.4.965.
[41] P. P. Roy and K. Roy, “On Some Aspects of Variable Selection for Partial Least Squares Regression Models,” QSAR Comb. Sci., vol. 27, no. 3, pp. 302–313, Mar. 2008, doi: 10.1002/qsar.200710043.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0