(Universitas Indonesia, Indonesia)(2) * Evi Yulianti
(Universitas Indonesia, Indonesia)*corresponding author
AbstractAcademic expert finding has numerous advantages, such as: finding paper-reviewers, research collaboration, enhancing knowledge transfer, etc. Especially, for research collaboration, researchers tend to seek collaborators who share similar backgrounds or with the same native languages. Despite its importance, academic expert findings remain relatively unexplored within the context of Indonesian language. Recent studies have primarily relied on static word embedding techniques such as Word2Vec to match documents with relevant expertise areas. However, Word2Vec is unable to capture the varying meanings of words in different contexts. To address this research gap, this study employs Bidirectional Encoder Representations from Transformers (BERT), a state-of-the-art contextual embedding model. This paper aims to examine the effectiveness of BERT on the task of academic expert finding. The proposed model in this research consists of three variations of BERT, namely IndoBERT (Indonesian BERT), mBERT (Multilingual BERT), and SciBERT (Scientific BERT), which will be compared to a static embedding model using Word2Vec. Two approaches were employed to rank experts using the BERT variations: feature-based and fine-tuning. We found that the IndoBERT model outperforms the baseline by 6–9% when utilizing the feature-based approach and shows an improvement of 10–18% with the fine-tuning approach. Our results proved that the fine-tuning approach performs better than the feature-based approach, with an improvement of 1–5%. It concludes by using IndoBERT, this research has shown an improved effectiveness in the academic expert finding within the context of Indonesian language.
KeywordsAcademic expert finding; Contextual embedding; Static embedding; BERT; Word2Vec
|
DOIhttps://doi.org/10.26555/ijain.v10i2.1497 |
Article metricsAbstract views : 224 | PDF views : 51 |
Cite |
Full Text Download
|
References
[1] O. Husain, N. Salim, R. A. Alias, S. Abdelsalam, and A. Hassan, “Expert Finding Systems: A Systematic Review,” Appl. Sci., vol. 9, no. 20, p. 4250, Oct. 2019, doi: 10.3390/app9204250.
[2] K. Balog, “Expertise Retrieval,” Found. Trends® Inf. Retr., vol. 6, no. 2–3, pp. 127–256, 2012, doi: 10.1561/1500000024.
[3] R. Gonçalves and C. F. Dorneles, “Automated Expertise Retrieval,” ACM Comput. Surv., vol. 52, no. 5, pp. 1–30, Sep. 2020, doi: 10.1145/3331000.
[4] M. I. M. Ishag, K. H. Park, J. Y. Lee, and K. H. Ryu, “A Pattern-Based Academic Reviewer Recommendation Combining Author-Paper and Diversity Metrics,” IEEE Access, vol. 7, pp. 16460–16475, 2019, doi: 10.1109/ACCESS.2019.2894680.
[5] Z. Ban and L. Liu, “CICPV: A New Academic Expert Search Model,” in 2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA), Mar. 2016, vol. 2016-May, pp. 47–52, doi: 10.1109/AINA.2016.14.
[6] M. Neshati, S. H. Hashemi, and H. Beigy, “Expertise Finding in Bibliographic Network: Topic Dominance Learning Approach,” IEEE Trans. Cybern., vol. 44, no. 12, pp. 2646–2657, Dec. 2014, doi: 10.1109/TCYB.2014.2312614.
[7] D. Liu, W. Xu, W. Du, and F. Wang, “How to Choose Appropriate Experts for Peer Review: An Intelligent Recommendation Method in a Big Data Context,” Data Sci. J., vol. 14, no. 0, p. 16, May 2015, doi: 10.5334/dsj-2015-016.
[8] S. Knop, R. Merchel, and J. Poeppelbuss, “Author Collaboration in Ten Years of IPS2: A Bibliometric Analysis,” Procedia CIRP, vol. 83, pp. 22–27, Jan. 2019, doi: 10.1016/j.procir.2019.03.092.
[9] R. Saptono, H. Setiadi, T. Sulistyoningrum, and E. Suryani, “Examiners Recommendation System at Proposal Seminar of Undergraduate Thesis by Using Content- based Filtering,” in 2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS), Oct. 2018, pp. 295–299, doi: 10.1109/ICACSIS.2018.8618224.
[10] S. Al Hakim, D. I. Sensuse, I. Budi, I. M. I. Subroto, and A. H. A. M. Siagian, “Expert retrieval based on local journals metadata to drive small-medium industries (SMI) collaboration for product innovation,” Soc. Netw. Anal. Min., vol. 13, no. 1, p. 68, Apr. 2023, doi: 10.1007/s13278-023-01044-5.
[11] T. V. Rampisela and E. Yulianti, “Academic Expert Finding in Indonesia using Word Embedding and Document Embedding: A Case Study of Fasilkom UI,” in 2020 8th International Conference on Information and Communication Technology (ICoICT), Jun. 2020, pp. 1–6, doi: 10.1109/ICoICT49345.2020.9166249.
[12] T. V. Rampisela and E. Yulianti, “Semantic-Based Query Expansion for Academic Expert Finding,” in 2020 International Conference on Asian Language Processing (IALP), Dec. 2020, pp. 34–39, doi: 10.1109/IALP51396.2020.9310492.
[13] N. A. Smith and P. G. Allen, “Contextual Word Representations: A Contextual Introduction,” arxiv Comput. Sci., p. 15, 2020. [Online]. Available at: https://arxiv.org/abs/1902.06006.
[14] K. Ethayarajh, “How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 55–65, doi: 10.18653/v1/D19-1006.
[15] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North, 2019, no. Mlm, pp. 4171–4186, doi: 10.18653/v1/N19-1423.
[16] B. Wilie et al., “IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding,” in Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, 2020, pp. 843–857. [Online]. Available: https://aclanthology.org/2020.aacl-main.85.
[17] I. Beltagy, K. Lo, and A. Cohan, “SciBERT: A Pretrained Language Model for Scientific Text,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 3613–3618, doi: 10.18653/v1/D19-1371.
[18] R. C. Lima and R. L. T. Santos, “On Extractive Summarization for Profile-centric Neural Expert Search in Academia,” in Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul. 2022, pp. 2331–2335, doi: 10.1145/3477495.3531713.
[19] C. Wu, F. Wu, T. Qi, X. Cui, and Y. Huang, “Attentive Pooling with Learnable Norms for Text Representation,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 2961–2970, doi: 10.18653/v1/2020.acl-main.267.
[20] A. Jafari, “Comparison Study Between Token Classification and Sequence Classification In Text Classification,” arxiv Comput. Sci., p. 11, 2022. [Online]. Available at: https://arxiv.org/abs/2211.13899.
[21] C. Bass, B. Benefield, D. Horn, and R. Morones, “Increasing Robustness in Long Text Classifications Using Background Corpus Knowledge for Token Selection.,” SMU Data Sci. Rev., vol. 2, no. 3, p. 10, Jan. 2020. [Online]. Available at: https://scholar.smu.edu/datasciencereview/vol2/iss3/10.
[22] C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge University Press, p. 506, 2008, doi: 10.1017/CBO9780511809071.
[23] J. Urbano, H. Lima, and A. Hanjalic, “Statistical Significance Testing in Information Retrieval,” in Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul. 2019, pp. 505–514, doi: 10.1145/3331184.3331259.
[24] D. Rau and J. Kamps, “How Different are Pre-trained Transformers for Text Ranking?,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13186 LNCS, Springer Science and Business Media Deutschland GmbH, 2022, pp. 207–214, doi: 10.1007/978-3-030-99739-7_24.
[25] C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to Fine-Tune BERT for Text Classification?,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11856 LNAI, Springer, 2019, pp. 194–206, doi: 10.1007/978-3-030-32381-3_16.
[26] K. N. Elmadani, M. Elgezouli, and A. Showk, “BERT Fine-tuning For Arabic Text Summarization,” arxiv Comput. Sci., p. 4, 2020. [Online]. Available at: https://arxiv.org/abs/2004.14135.
[27] T. Tang, X. Tang, and T. Yuan, “Fine-Tuning BERT for Multi-Label Sentiment Analysis in Unbalanced Code-Switching Text,” IEEE Access, vol. 8, pp. 193248–193256, 2020, doi: 10.1109/ACCESS.2020.3030468.
[28] E. Wallace, Y. Wang, S. Li, S. Singh, and M. Gardner, “Do NLP Models Know Numbers? Probing Numeracy in Embeddings,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 5306–5314, doi: 10.18653/v1/D19-1534.
[29] A. Askari, A. Abolghasemi, G. Pasi, W. Kraaij, and S. Verberne, “Injecting the BM25 Score as Text Improves BERT-Based Re-rankers,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13980 LNCS, Springer Science and Business Media Deutschland GmbH, 2023, pp. 66–83, doi: 10.1007/978-3-031-28244-7_5.
[30] A. Askari, A. Abolghasemi, G. Pasi, W. Kraaij, and S. Verberne, “Injecting the Score of the First-stage Retriever as Text Improves BERT-Based Re-rankers,” in European Conference on Information Retrieval, Oct. 2023, pp. 1–27, doi: 10.21203/rs.3.rs-3398657/v1.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0