(Ministry of Higher Education, Iraq)(2) Safaa Obayes Al-Mamory
(College of Business Informatics, University of Information Technology and Communications, Iraq)(3) Ali Hashim Al-Shakarchi
(College of Information Technology, University of Babylon, Iraq)*corresponding author
AbstractA large spectrum of classifiers has been described in the literature. One attractive classification technique is a Naïve Bayes (NB) which has been relayed on probability theory. NB has two major limitations: First, it requires to rescan the dataset and applying a set of equations each time to classify instances, which is an expensive step if a dataset is relatively large. Second, NB may remain challenging for non-statisticians to understand the deep work of a model. On the other hand, Rule-Based classifiers (RBCs) have used IF-THEN rules (henceforth, rule-set), which are more comprehensible and less complex for classification tasks. For elevating NB limitations, this paper presents a method for constructing a rule-set from the NB model, which serves as RBC. Experiments of the constructing rule-set have been conducted on (Iris, WBC, Vote) datasets. Coverage, Accuracy, M-Estimate, and Laplace are crucial evaluation metrics that have been projected to rule-set. In some datasets, the rule-set obtains significant accuracy results that reach 95.33 %, 95.17% for Iris and vote datasets, respectively. The constructed rule-set can mimic the classification capability of NB, provide a visual representation of the model, express rules infidelity with acceptable accuracy; an easier method to interpreting and adjusting from the original model. Hence, the rule-set will provide a comprehensible and lightweight model than NB itself.
|
DOIhttps://doi.org/10.26555/ijain.v7i1.578 |
Article metricsAbstract views : 691 | PDF views : 188 |
Cite |
Full Text Download
|
References
[1] S. Garg and S. Batra, "A novel ensembled technique for anomaly detection," Int. J. Commun. Syst., vol. 30, no. 11, p. e3248, 2017, doi: 10.1002/dac.3248.
[2] R. González Perea, E. Camacho Poyato, P. Montesinos, and J. A. Rodríguez Díaz, “Prediction of irrigation event occurrence at farm level using optimal decision trees,” Comput. Electron. Agric., vol. 157, pp. 173–180, 2019, doi: 10.1016/j.compag.2018.12.043.
[3] M. A. Ferrag, L. Maglaras, A. Ahmim, M. Derdour, and H. Janicke, "RDTIDS: Rules and Decision Tree-Based Intrusion Detection System for Internet-of-Things Networks," Futur. Internet, vol. 12, no. 3, p. 44, 2020, doi: 10.3390/fi12030044
[4] J. Nalepa and M. Kawulok, "Selecting training sets for support vector machines: a review," Artif. Intell. Rev., vol. 52, no. 2, pp. 857–900, 2019, doi: 10.1007/s10462-017-9611-1
[5] Q. Wu, Z. Ma, G. Xu, S. Li, and D. Chen, "A novel neural network classifier using beetle antennae search algorithm for pattern classification," IEEE access, vol. 7, pp. 64686–64696, 2019, doi: 10.1109/ACCESS.2019.2917526
[6] T. Alasalmi, J. Suutala, H. Koskimäki, and J. Röning, “Better Classifier Calibration for Small Data Sets,” arXiv Prepr. arXiv2002.10199, 2020, doi: 10.1145/3385656
[7] J. Li, M. Gao, and R. D'Agostino, "Evaluating classification accuracy for modern learning approaches," Stat. Med., vol. 38, no. 13, pp. 2477–2503, 2019, doi: 10.1002/sim.8103
[8] N. Barakat and J. Diederich, "Eclectic Rule-Extraction from Support Vector Machines," Eng. Technol., vol. 2, no. 1, pp. 331–334, 2006, doi: 10.1080/00029157.1964.10402393.
[9] D. Martens, B. B. Baesens, T. Van Gestel, and T. Van Gestel, "Decompositional rule extraction from support vector machines by active learning," IEEE Trans. Knowl. Data Eng., vol. 21, no. 2, pp. 178–191, 2009, doi: 10.1007/978-3-540-75390-2.
[10] C. Panigutti, R. Guidotti, A. Monreale, and D. Pedreschi, "Explaining multi-label black-box classifiers for health applications," in International Workshop on Health Intelligence, 2019, pp. 97–110, doi: 10.1109/TKDE.2008.131
[11] G. L. Pappa and A. Freitas, Automating the Design of Data Mining Algorithms, 2010, doi: 10.1007/978-3-642-02541-9, doi: 10.1007/978-3-030-24409-5_9
[12] O. Loyola-González, "Black-box vs. white-box: Understanding their advantages and weaknesses from a practical point of view," IEEE Access, vol. 7, pp. 154096–154113, 2019, doi: 10.1007/978-3-642-02541-9
[13] M. M. Saritas and A. Yasar, "Performance Analysis of ANN and Naive Bayes Classification Algorithm for Data Classification," Int. J. Intell. Syst. Appl. Eng., vol. 7, no. 2, pp. 88–91, 2019, doi: 10.1109/ACCESS.2019.2949286
[14] A. Sahasrabuddhe, S. Naikade, A. Ramaswamy, B. Sadliwala, and P. Futane, "Survey on Intrusion Detection System using Data Mining Techniques," Int. Res. J. Eng. Technol., vol. 4, no. 5, pp. 1780–1784, 2017, doi: ISSN: 2395 -0056, doi: 10.18201/ijisae.2019252786
[15] K. Kumar, R. Jha, and S. Afroz, "Data Mining Techniques for Intrusion Detection : A Review," Int. J. Adv. Researcg Comput. Commun. Enginering, vol. 3, no. 6, pp. 6938–6942, 2014, Available at: Google Scholar
[16] A. Alashqur, "A Novel Methodology for Constructing Rule-Based Naïve Bayesian Classifiers," Int. J. Comput. Sci. Inf. Technol., vol. 7, no. 1, pp. 139–151, 2015, doi: 10.5121/ijcsit.2015.7114.
[17] R. Andrews, J. Diederich, and A. B. Tickle, "Survey and critique of techniques for extracting rules from trained artificial neural networks," Knowledge-Based Syst., vol. 8, no. 6, pp. 373–389, 1995, doi: 10.1016/0950-7051(96)81920-4.
[18] D. Martens, B. Baesens, T. Van Gestel, and J. Vanthienen, "Comprehensible credit scoring models using rule extraction from support vector machines," Eur. J. Oper. Res., vol. 183, no. 3, pp. 1466–1476, 2007, doi: 10.1016/j.ejor.2006.04.051.
[19] G. Towell and J. W. Shavlik, "Interpretation of artificial neural networks: Mapping knowledge-based neural networks into rules," in Advances in neural information processing systems, 1992, pp. 977–984, Available at: Google Scholar
[20] L. Fu, "Rule learning by searching on adapted nets," Proc. 9th Natl. Conf. Artif. Intell. - Vol. 2, vol. 91, pp. 590–595, 1991. Available at: Google Scholar
[21] J. R. Quinlan, "Generating production rules from decision trees," in Proceedings of the Tenth International Joint Conference on Artificial Intelligence, 1987, vol. 30107, pp. 304–307. Available at: Google Scholar
[22] H. Núñez, C. Angulo, and A. Català, "Rule extraction from support vector machines.," in Esann, 2002, pp. 107–112. Available at: Google Scholar
[23] B. Śnieżyński, "Converting a Naive Bayes Models with Multi-valued Domains into Sets of Rules," Int. Conf. Database Expert Syst. Appl., vol. 229, pp. 634–643, 2006, doi: 10.1007/11827405_62.
[24] J. Diederich and N. Barakat, "Hybrid rule-extraction from support vector machines," in 2004 IEEE Conference on Cybernetics and Intelligent Systems, 2004, vol. vol.2, pp. 1270–1275, doi: 10.1109/ICCIS.2004.1460774
[25] R. Setiono, C. Mues, and B. Baesens, "Risk management and regulatory compliance: A data mining framework based on neural network rule extraction," ICIS 2006 Proc. - Twenty Seventh Int. Conf. Inf. Syst., pp. 71–86, 2006. Available at: Google Scholar
[26] M. W. Craven and J. W. Shavlik, "Extracting Tree-Structured Representations of Trained Networks," Adv. Neural Inf. …, vol. 8, p. 7, 1996. Available at: Google Books
[27] U. Markowska-Kaczmar and M. Chumieja, “Discovering the mysteries of neural networks,” Int. J. Hybrid Intell. Syst., vol. 1, no. 3–4, pp. 153–163, 2004, doi: 10.3233/HIS-2004-13-404
[28] F. Chen, "Learning Accurate and Understandable Rules From Svm Classifiers," Ph.D. thesis, Science: School of Computing Science, 2004. Available at: Google Scholar
[29] S. Xu, “Bayesian Naïve Bayes classifiers to text classification,” J. Inf. Sci., vol. 44, no. 1, pp. 48–59, 2018, doi: 10.1177/0165551516677946.
[30] S. A. Mostafa et al., "Examining multiple feature evaluation and classification methods for improving the diagnosis of Parkinson's disease," Cogn. Syst. Res., vol. 54, pp. 90–99, 2019, doi: 10.1016/j.cogsys.2018.12.004
[31] M. Goldszmidt, "Bayesian Network Classifiers," Wiley Encycl. Oper. Res. Manag. Sci., vol. 29, no. 2–3, pp. 131–163, 2011, doi: 10.1002/9780470400531.eorms0099, doi: 10.1002/9780470400531.eorms0099
[32] M. Možina, J. Demšar, M. Kattan, and B. Zupan, "Nomograms for Visualization of Naive Bayesian Classifier," in European Conference on Principles of Data Mining and Knowledge Discovery, 2010, vol. 3202, pp. 337–348, doi: 10.1007/978-3-540-30116-5_32.
[33] I. Technologies, A. S. Kapse, V. P. Kshirsagar, A. Kapse, and M. B. Nagori, "A Survey on Intrusion Network Detection System Using Data Mining Techniques," vol. 2, no. 3, pp. 1253–1256, 2011. Available at: ijcsit.com
[34] M. R. Karlsen and S. Moschoyiannis, "Learning condition--action rules for personalised journey recommendations," in International Joint Conference on Rules and Reasoning, 2018, pp. 293–301, doi: 10.1007/978-3-319-99906-7_21
[35] A. R. Webb and K. D. Copsey, Statistical Pattern Recoginiton. WILEY, 2011, doi: 10.1002/9781119952954
[36] C. L. Blake and C. J. Merz, "UCI Repository of machine learning databases," University of California. Accessed, p. http://archive.ics.uci.edu/ml/, 2018. Available at: Google Scholar
[37] E. Frank, M. A. Hall, and I. H. Witten, The WEKA workbench, 2017, doi: 10.1016/b978-0-12-804291-5.00024-6.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0