(Universitas Gunadarma, Indonesia)(2) Sarifuddin Madenda
(Universitas Gunadarma, Indonesia)(3) Setia Wirawan
(Universitas Gunadarma, Indonesia)(4) Ruddy J. Suhatril
(Universitas Gunadarma, Indonesia)*corresponding author
AbstractScientific articles now have multidisciplinary content. These make it difficult for researchers to find out relevant information. Some submissions are irrelevant to the journal's discipline. Categorizing articles and assessing their relevance can aid researchers and journals. Existing research still focuses on single-category predictive outcomes. Therefore, this research takes a new approach by applying a multidisciplinary classification for Indonesian scientific article abstracts using a pre-trained BERT model, showing the relevance between each category in an abstract. The dataset used was 9,000 abstracts with 9 disciplinary categories. On the dataset, text preprocessing is performed. The classification model was built by combining the pre-trained BERT model with Artificial Neural Network. Fine-tuning the hyperparameters is done to determine the most optimal hyperparameter combination for the model. The hyperparameters consist of batch size, learning rate, number of epochs, and data ratio. The best hyperparameter combination is a learning rate of 1e-5, batch size 32, epochs 3, and data ratio 9:1, with a validation accuracy value of 90.8%. The confusion matrix results of the model are compared with the confusion matrix results by experts. In this case, the highest accuracy result obtained by the model is 99.56%. A software prototype used the most accurate model to classify new data, displaying the top two prediction probabilities and the dominant category. This research produces a model that can be used to solve Indonesian text classification-related problems.
KeywordsAbstract; BERT; Classification; Hyperparameter-Tuning; Multidiciplinary
|
DOIhttps://doi.org/10.26555/ijain.v9i2.1051 |
Article metricsAbstract views : 655 | PDF views : 186 |
Cite |
Full Text Download
|
References
[1] F. R. Lumbanraja, E. Fitri, Ardiansyah, A. Junaidi, and R. Prabowo, “Abstract Classification Using Support Vector Machine Algorithm (Case Study: Abstract in a Computer Science Journal),” J. Phys. Conf. Ser., vol. 1751, no. 1, pp. 0–12, 2021, doi: 10.1088/1742-6596/1751/1/012042.
[2] A. KP and J. Anitha, “Plant disease classification using deep learning,” in 2021 3rd International Conference on Signal Processing and Communication (ICPSC), May 2021, pp. 407–411, doi: 10.1109/ICSPC51351.2021.9451696.
[3] I. N. Khasanah, “Sentiment Classification Using fastText Embedding and Deep Learning Model,” Procedia CIRP, vol. 189, pp. 343–350, 2021, doi: 10.1016/j.procs.2021.05.103.
[4] I. M. Fadhil and Y. Sibaroni, “Topic Classification in Indonesian-language Tweets using Fast-Text Feature Expansion with Support Vector Machine (SVM),” in 2022 International Conference on Data Science and Its Applications (ICoDSA), Jul. 2022, pp. 214–219, doi: 10.1109/ICoDSA55874.2022.9862899.
[5] R. Adipradana, B. P. Nayoga, R. Suryadi, and D. Suhartono, “Hoax analyzer for Indonesian news using RNNs with fasttext and glove embeddings,” Bull. Electr. Eng. Informatics, vol. 10, no. 4, pp. 2130–2136, Aug. 2021, doi: 10.11591/eei.v10i4.2956.
[6] R. Kusumaningrum, I. Z. Nisa, R. P. Nawangsari, and A. Wibowo, “Sentiment analysis of Indonesian hotel reviews: from classical machine learning to deep learning,” Int. J. Adv. Intell. Informatics, vol. 7, no. 3, pp. 292–303, Nov. 2021, doi: 10.26555/ijain.v7i3.737.
[7] M. S. David and S. Renjith, “Comparison of word embeddings in text classification based on RNN and CNN,” IOP Conf. Ser. Mater. Sci. Eng., vol. 1187, no. 1, p. 012029, Sep. 2021, doi: 10.1088/1757-899X/1187/1/012029.
[8] M. I. Alfarizi, L. Syafaah, and M. Lestandy, “Emotional Text Classification Using TF-IDF (Term Frequency-Inverse Document Frequency) And LSTM (Long Short-Term Memory),” JUITA J. Inform., vol. 10, no. 2, p. 225, Nov. 2022, doi: 10.30595/juita.v10i2.13262.
[9] K. Boonchuay, “Sentiment Classification Using Text Embedding for Thai Teaching Evaluation,” Appl. Mech. Mater., vol. 886, pp. 221–226, Jan. 2019, doi: 10.4028/www.scientific.net/AMM.886.221.
[10] J. Devlin, M.-W. Chang, K. Lee, K. T. Google, and A. I. Language, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Naacl-Hlt 2019, no. Mlm, pp. 1-16, 2019, doi: 10.48550/arXiv.1810.04805.
[11] M. Munikar, S. Shakya, and A. Shrestha, “Fine-grained Sentiment Classification using BERT,” Int. Conf. Artif. Intell. Transform. Bus. Soc. AITB 2019, vol. 1, pp. 1–5, 2019, doi: 10.1109/AITB48515.2019.8947435.
[12] S. Abdul, Y. Qiang, S. Basit, and W. Ahmad, “Using BERT for Checking the Polarity of Movie Reviews,” Int. J. Comput. Appl., vol. 177, no. 21, pp. 37–41, 2019, doi: 10.5120/ijca2019919675.
[13] W. Maharani, “Sentiment Analysis during Jakarta Flood for Emergency Responses and Situational Awareness in Disaster Management using BERT,” 2020 8th Int. Conf. Inf. Commun. Technol. ICoICT 2020, pp. 1–5, 2020, doi: 10.1109/ICoICT49345.2020.9166407.
[14] J. Ravi and S. Kulkarni, “Text embedding techniques for efficient clustering of twitter data,” Evol. Intell., pp. 1-11, Feb. 2023, doi: 10.1007/s12065-023-00825-3.
[15] M. Khadhraoui, H. Bellaaj, M. Ben Ammar, H. Hamam, and M. Jmaiel, “Survey of BERT-Base Models for Scientific Text Classification: COVID-19 Case Study,” Appl. Sci., vol. 12, no. 6, p. 2891, Mar. 2022, doi: 10.3390/app12062891.
[16] X. Chen, P. Cong, and S. Lv, “A Long-Text Classification Method of Chinese News Based on BERT and CNN,” IEEE Access, vol. 10, pp. 34046–34057, 2022, doi: 10.1109/ACCESS.2022.3162614.
[17] G. Danilov, T. Ishankulov, K. Kotik, Y. Orlov, M. Shifrin, and A. Potapov, “The Classification of Short Scientific Texts Using Pretrained BERT Model,” vol. 281, pp. 83-87, 2021, doi: 10.3233/SHTI210125.
[18] I. M. Rabbimov and S. S. Kobilov, “Multi-Class Text Classification of Uzbek News Articles using Machine Learning,” in Journal of Physics: Conference Series, May 2020, vol. 1546, no. 1, pp. 012097, doi: 10.1088/1742-6596/1546/1/012097.
[19] A. Bogdanchikov, D. Ayazbayev, and I. Varlamis, “Classification of Scientific Documents in the Kazakh Language Using Deep Neural Networks and a Fusion of Images and Text,” Big Data Cogn. Comput., vol. 6, no. 4, p. 123, Oct. 2022, doi: 10.3390/bdcc6040123.
[20] A. Barua, O. Sharif, and M. M. Hoque, “Multi-class Sports News Categorization using Machine Learning Techniques: Resource Creation and Evaluation,” Procedia Comput. Sci., vol. 193, pp. 112–121, 2021, doi: 10.1016/j.procs.2021.11.002.
[21] D. Ali, M. M. S. Missen, and M. Husnain, “Multiclass Event Classification from Text,” Sci. Program., vol. 2021, pp. 1–15, Jan. 2021, doi: 10.1155/2021/6660651.
[22] Y. A. Putra and M. L. Khodra, “Deep learning and distributional semantic model for Indonesian tweet categorization,” in 2016 International Conference on Data and Software Engineering (ICoDSE), 2016, pp. 1–6, doi: 10.1109/ICODSE.2016.7936108.
[23] A. K. Uysal and S. Gunal, “The impact of preprocessing on text classification,” Inf. Process. Manag., vol. 50, no. 1, pp. 104–112, Jan. 2014, doi: 10.1016/j.ipm.2013.08.006.
[24] D. Haryalesmana and M. Wieriks, “Indonesian Stopword Corpus,” 2016. [Online]. Available at: https://github.com/masdevid/ID-Stopwords
[25] F. Z. Tala, “A Study of Stemming Effects on Information Retrieval in Bahasa Indonesia,” Institute for Logic, Language and Computation. Universiteit van Amsterdam, The Netherlands., 2003. [Online]. Available at: https://eprints.illc.uva.nl/id/eprint/740/1/MoL-2003-02.text.pdf.
[26] W. C, “BERT-base-indonesian-522M,” Hugging Face, 2021. [Online]. Available at: https://huggingface.co/cahya/bert-base-indonesian-522M.
[27] A. Vaswani et al., “Attention is all you need,” in NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 6000–6010, doi: 10.48550/arXiv.1706.03762.
[28] Y. Bai, “RELU-Function and Derived Function Review,” SHS Web Conf., vol. 144, p. 02006, Aug. 2022, doi: 10.1051/shsconf/202214402006.
[29] S. Pothuganti, “Review on over-fitting and under-fitting problems in Machine Learning and solutions,” Int. J. Adv. Res. Electr. Electron. Instrum. Eng., vol. 7, pp. 3692–3695, Sep. 2018. [Online]. Available at: http://www.ijareeie.com/upload/2018/september/11A_PS_NC.PDF.
[30] I. Kandel and M. Castelli, “The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset,” ICT Express, vol. 6, no. 4, pp. 312–315, Dec. 2020, doi: 10.1016/j.icte.2020.04.010.
[31] H. Jindal, N. Sardana, and R. Mehta, “Analyzing Performance of Deep Learning Techniques for Web Navigation Prediction,” Procedia Comput. Sci., vol. 167, pp. 1739–1748, 2020, doi: 10.1016/j.procs.2020.03.384.
[32] M. Hossin and M. N. Sulaiman, “A Review on Evaluation Metrics for Data Classification Evaluations,” Int. J. Data Min. Knowl. Manag. Process, vol. 5, no. 2, pp. 01–11, Mar. 2015, doi: 10.5121/ijdkp.2015.5201.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0