(Universitas Pembangunan Nasional "Veteran" Yogyakarta Indonesia &
FTMK UTeM Melaka Malaysia, Indonesia)(2) Halizah Basiron
(Fakulti Teknologi Maklumat dan Komunikasi, Universiti Teknikal Malaysia Melaka, Malaysia)(3) Noor Fazilla Abd Yusof
(Fakulti Teknologi Maklumat dan Komunikasi, Universiti Teknikal Malaysia Melaka, Malaysia)*corresponding author
AbstractSupervised sentiment analysis ideally uses a fully labeled data set for modeling. However, this ideal condition requires a struggle in the label annotation process. Semi-supervised learning (SSL) has emerged as a promising method to avoid time-consuming and expensive data labeling without reducing model performance. However, the research on SSL is still limited and its performance needs to be improved. Thus, this study aims to create a new SSL-Model for sentiment analysis. The Ensemble Classifier SSL model for sentiment classification is introduced. The research went through pre-processing, vectorization, and feature extraction using TF-IDF and n-grams. Support Vector Machine (SVM) or Random Forest for tokenization was used to separate unigram, bigram, and trigram in model generation. Then, the outputs of these models were combined using stacking ensemble approach. Accuracy and F1-score were used for the evaluation. IMDB datasets and US Airlines were used to test the new SSL models. The conclusion is that the sentiment annotation accuracy is highly dependent on the suitability of the dataset with the machine learning algorithm. In IMDB dataset, which consists of two classes, it is better to use SVM. In the US Airlines consisting of three classes, SVM is better at improving the model performance against the baseline, but RF is better at achieving the baseline performance even though it fails to maintain the model performance.
KeywordsEnsemble Multi-classifier; Semi-supervised; Sentiment Analysis; SVM; Random Forest
|
DOIhttps://doi.org/10.26555/ijain.v8i3.929 |
Article metricsAbstract views : 747 | PDF views : 198 |
Cite |
Full Text Download
|
References
[1] P. P. Patil, S. Phansalkar, and V. V. Kryssanov, Topic modelling for aspect-level sentiment analysis, vol. 828. Springer Singapore, 2019. doi: 10.1007/978-981-13-1610-4_23.
[2] V. L. Shan Lee, K. H. Gan, T. P. Tan, and R. Abdullah, “Semi-supervised learning for sentiment classification using small number of labeled data,” Procedia Comput. Sci., vol. 161, pp. 577–584, 2019, doi: 10.1016/j.procs.2019.11.159.
[3] Z. Zhang, J. Han, J. Deng, X. Xu, F. Ringeval, and B. Schuller, “Leveraging Unlabeled Data for Emotion Recognition with Enhanced Collaborative Semi-Supervised Learning,” IEEE Access, vol. 6, pp. 22196–22209, 2018, doi: 10.1109/ACCESS.2018.2821192.
[4] A. Alessa and M. F. B, Analysis Features and TF-IDF Weighting, vol. 2, no. Cdc. Springer International Publishing, 2018. doi: 10.1007/978-3-319-96136-1_15.
[5] V. Kumar and B. Subba, “A Tfidfvectorizer and SVM based sentiment analysis framework for text data corpus,” 26th Natl. Conf. Commun. NCC 2020, pp. 1–6, 2020, doi: 10.1109/NCC48643.2020.9056085.
[6] A. U. Rehman, A. K. Malik, B. Raza, and W. Ali, “A Hybrid CNN-LSTM Model for Improving Accuracy of Movie Reviews Sentiment Analysis,” Multimed. Tools Appl., vol. 78, no. 18, pp. 26597–26613, Sep. 2019, doi: 10.1007/S11042-019-07788-7.
[7] F. ÖZYURT and M. HUSSEİN, “A New Technique for Sentiment Analysis System Based on Deep Learning Using Chi-Square Feature Selection Methods,” Balk. J. Electr. Comput. Eng., vol. 9, no. 4, pp. 320–326, 2021, Accessed: Jan. 04, 2023. [Online]. Available: https://dergipark.org.tr/en/pub/bajece/issue/65264/887339
[8] A. Al-Laith, M. Shahbaz, H. F. Alaskar, and A. Rehmat, “AraSenCorpus: A Semi-Supervised Approach for Sentiment Annotation of a Large Arabic Text Corpus,” Appl. Sci., vol. 11, no. 5, pp. 1–19, 2021, doi: 10.3390/app11052434.
[9] A. Alqarafi, A. Adeel, A. Hawalah, K. Swingler, and A. Hussain, A Semi-supervised Corpus Annotation for Saudi Sentiment Analysis Using Twitter, vol. 10989 LNAI. Springer International Publishing, 2018. doi: 10.1007/978-3-030-00563-4_57.
[10] O. Al-Harbi, “Classifying sentiment of dialectal arabic reviews: A semi-supervised approach,” Int. Arab J. Inf. Technol., vol. 16, no. 6, pp. 995–1002, 2019. Available at : https://iajit.org/portal/index.php/archive/volume-16-2019/november-2019-no-6/item/136-classifying-sentiment-of-dialectal-arabic-reviews-a-semi-supervised-approach
[11] V. D. H. de Carvalho, T. C. C. Nepomuceno, and A. P. C. S. Costa, An Automated Corpus Annotation Experiment in Brazilian Portuguese for Sentiment Analysis in Public Security, vol. 384 LNBIP. Springer International Publishing, 2020. doi: 10.1007/978-3-030-46224-6_8.
[12] V. Balakrishnan, P. Y. Lok, and H. Abdul Rahim, “A semi-supervised approach in detecting sentiment and emotion based on digital payment reviews,” J. Supercomput., vol. 77, no. 4, pp. 3795–3810, 2021, doi: 10.1007/s11227-020-03412-w.
[13] Y. Han, Y. Liu, and Z. Jin, “Sentiment analysis via semi-supervised learning: a model based on dynamic threshold and multi-classifiers,” Neural Comput. Appl., vol. 32, no. 9, pp. 5117–5129, 2020, doi: 10.1007/s00521-018-3958-3.
[14] A. Naresh and P. Venkata Krishna, “An efficient approach for sentiment analysis using machine learning algorithm,” Evol. Intell., vol. 14, no. 2, pp. 725–731, 2021, doi: 10.1007/s12065-020-00429-1.
[15] S. Tiun, U. A. Mokhtar, S. H. Bakar, and S. Saad, “Classification of functional and non-functional requirement in software requirement using Word2vec and fast Text,” J. Phys. Conf. Ser., vol. 1529, no. 4, 2020, doi: 10.1088/1742-6596/1529/4/042077.
[16] A. Rane and A. Kumar, “Sentiment Classification System of Twitter Data for US Airline Service Analysis,” in Proceedings - International Conference on Computer Software and Applications, 2018, pp. 769–773. doi: 10.1109/COMPSAC.2018.00114.
[17] R. S. Kumar, A. F. Saviour Devaraj, M. Rajeswari, E. G. Julie, Y. H. Robinson, and V. Shanmuganathan, “Exploration of sentiment analysis and legitimate artistry for opinion mining,” Multimed. Tools Appl., 2021, doi: 10.1007/s11042-020-10480-w.
[18] Y. Pan, Z. Chen, Y. Suzuki, F. Fukumoto, and H. Nishizaki, “Sentiment analysis using semi-supervised learning with few labeled data,” Proc. - 2020 Int. Conf. Cyberworlds, CW 2020, pp. 231–234, 2020, doi: 10.1109/CW49994.2020.00044.
[19] M. A. Azim and M. H. Bhuiyan, “Text to emotion extraction using supervised machine learning techniques,” Telkomnika (Telecommunication Comput. Electron. Control., vol. 16, no. 3, pp. 1394–1401, 2018, doi: 10.12928/TELKOMNIKA.v16i3.8387.
[20] S. Mitra and M. Jenamani, “SentiCon: A Concept Based Feature Set for Sentiment Analysis,” in 2018 13th International Conference on Industrial and Information Systems, ICIIS 2018 - Proceedings, 2018, no. 978, pp. 246–250. doi: 10.1109/ICIINFS.2018.8721408.
[21] P. H. Prastyo, I. Ardiyanto, and R. Hidayat, “Indonesian Sentiment Analysis: An Experimental Study of Four Kernel Functions on SVM Algorithm with TF-IDF,” 2020 Int. Conf. Data Anal. Bus. Ind. W. Towar. a Sustain. Econ. ICDABI 2020, 2020, doi: 10.1109/ICDABI51230.2020.9325685.
[22] S. S. M. M. Rahman, K. B. M. B. Biplob, M. H. Rahman, K. Sarker, and T. Islam, An investigation and evaluation of N-gram, TF-IDF and ensemble methods in sentiment classification, vol. 325 LNICST, no. August. Springer International Publishing, 2020. doi: 10.1007/978-3-030-52856-0_31.
[23] R. Hendrawan, Adiwijaya, and S. Al Faraby, “Multilabel Classification of Hate Speech and Abusive Words on Indonesian Twitter Social Media,” 2020 Int. Conf. Data Sci. Its Appl. ICoDSA 2020, 2020, doi: 10.1109/ICoDSA50139.2020.9212962.
[24] M. Aufar, R. Andreswari, and D. Pramesti, “Sentiment Analysis on Youtube Social Media Using Decision Tree and Random Forest Algorithm: A Case Study,” 2020 Int. Conf. Data Sci. Its Appl. ICoDSA 2020, 2020, doi: 10.1109/ICoDSA50139.2020.9213078.
[25] M. A. Fauzi, “Random forest approach fo sentiment analysis in Indonesian language,” Indones. J. Electr. Eng. Comput. Sci., vol. 12, no. 1, pp. 46–50, 2018, doi: 10.11591/ijeecs.v12.i1.pp46-50.
[26] A. R. Alaei, S. Becken, and B. Stantic, “Sentiment Analysis in Tourism: Capitalizing on Big Data,” J. Travel Res., vol. 58, no. 2, pp. 175–191, 2019, doi: 10.1177/0047287517747753.
[27] M. A. Fauzi, “Word2Vec model for sentiment analysis of product reviews in Indonesian language,” Int. J. Electr. Comput. Eng., vol. 9, no. 1, pp. 525–530, 2019, doi: 10.11591/ijece.v9i1.pp525-530.
[28] Y. Li, Y. Lv, S. Wang, J. Liang, J. Li, and X. Li, “Cooperative hybrid semi-supervised learning for text sentiment classification,” Symmetry (Basel)., vol. 11, no. 2, pp. 1–17, 2019, doi: 10.3390/sym11020133.
[29] J. Liang, R. Li, and Q. Jin, “Semi-supervised Multi-modal Emotion Recognition with Cross-Modal Distribution Matching,” MM 2020 - Proc. 28th ACM Int. Conf. Multimed., pp. 2852–2861, 2020, doi: 10.1145/3394171.3413579.
[30] S. Zhang et al., “Combining cross-modal knowledge transfer and semi-supervised learning for speech emotion recognition,” Knowledge-Based Syst., vol. 229, p. 107340, 2021, doi: 10.1016/j.knosys.2021.107340.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0