(Dian Nuswantoro University, Semarang, Indonesia)(2) Edi Noersasongko
(Dian Nuswantoro University, Semarang, Indonesia)(3) Pulung Nurtantio Andono
(Dian Nuswantoro University, Semarang, Indonesia)(4) Mochammad Arief Soeleman
(Dian Nuswantoro University Semarang, Indonesia)*corresponding author
AbstractConvolutional Neural Networks (CNNs) perform well compared to other deep learning models in image recognition, especially in handwritten alphabetic numeral datasets. CNN's challenging task is to find an architecture with the right hyperparameters. Usually, this activity is done by trial and error. A genetic algorithm (GA) has been widely used for automatic hyperparameter optimization. However, the original GA with fixed chromosome length allows for suboptimal solution results because CNN has a variable number of hyperparameters depending on the depth of the model. Previous work proposed variable chromosome lengths to overcome the drawbacks of native GA. This paper proposes a variable length GA by adding global hyperparameters, namely optimizer and learning speed, to systematically and automatically tune CNN hyperparameters to improve performance. We optimize seven hyperparameters, such as the learning rate. Optimizer, kernel, filter, activation function, number of layers and pooling. The experimental results show that a population of 25 produces the best fitness value and average fitness. In addition, the comparison results show that the proposed model is superior to the basic model based on accuracy. The experimental results show that the proposed model is about 99.18% higher than the baseline model.
KeywordsGenetic Algorithms, Hyperparameter Optimization, Convolutional Neural Networks, Handwritten Digit Recognition
|
DOIhttps://doi.org/10.26555/ijain.v9i1.881 |
Article metricsAbstract views : 926 | PDF views : 231 |
Cite |
Full Text Download
|
References
[1] L. M. Seng, B. B. C. Chiang, Z. A. A. Salam, G. Y. Tan, and H. T. Chai, “MNIST handwritten digit recognition with different CNN architectures,” J. Appl. Technol. Innov., vol. 5, no. 1, pp. 7–10, 2021. Available at : Google Scholar.
[2] S. Ali, Z. Shaukat, M. Azeem, Z. Sakhawat, T. Mahmood, and K. ur Rehman, “An efficient and improved scheme for handwritten digit recognition based on convolutional neural network,” SN Appl. Sci., vol. 1, no. 9, pp. 1–9, 2019, doi: 10.1007/s42452-019-1161-5.
[3] M. H. Abed, A. H. I. Al-Rammahi, and M. J. Radif, “Real-Time Color Image Classification Based On Deep Learning Network,” J. Southwest Jiaotong Univ., vol. 54, no. 5, 2019, doi: 10.35741/issn.0258-2724.54.5.23.
[4] M. Zohra and D. Rajeswara Rao, “A comprehensive data analysis on handwritten digit recognition using machine learning approach,” Int. J. Innov. Technol. Explor. Eng., vol. 8, no. 6, pp. 1449–1453, 2019. doi : 10.17509/ijost.v3i1.10795.
[5] K. Davoudi and P. Thulasiraman, “Evolving convolutional neural network parameters through the genetic algorithm for the breast cancer classification problem,” Simulation, vol. 97, no. 8, pp. 511–527, 2021, doi: 10.1177/0037549721996031.
[6] A. N. I. Hui, A. B. Huddin, M. F. Ibrahim, F. H. Hashim, and S. A. Samad, “GA-deep neural network optimization for image classification,” Int. J. Adv. Trends Comput. Sci. Eng., vol. 8, no. 1.6 Special Issue, pp. 238–245, 2019, doi: 10.30534/ijatcse/2019/3681.62019.
[7] T. N. Fatyanosa and M. Aritsugi, “An Automatic Convolutional Neural Network Optimization Using a Diversity-Guided Genetic Algorithm,” IEEE Access, vol. 9, pp. 91410–91426, 2021, doi: 10.1109/ACCESS.2021.3091729.
[8] I. Priyadarshini and C. Cotton, “A novel LSTM–CNN–grid search-based deep neural network for sentiment analysis,” J. Supercomput., vol. 77, no. 12, pp. 13911–13932, 2021, doi: 10.1007/s11227-021-03838-w.
[9] A. Aghaebrahimian and M. Cieliebak, “Hyperparameter tuning for deep learning in natural language processing,” CEUR Workshop Proc., vol. 2458, 2019. doi : 10.21256/zhaw-18993.
[10] F. M. Talaat and S. A. Gamel, “RL based hyper-parameters optimization algorithm (ROA) for convolutional neural network,” J. Ambient Intell. Humaniz. Comput., 2022, doi: 10.1007/s12652-022-03788-y.
[11] S. Albahli, F. Alhassan, W. Albattah, and R. Ullah, “Handwritten digit recognition: Hyperparameters-based analysis,” Appl. Sci., vol. 10, no. 17, 2020, doi: 10.3390/app10175988.
[12] M. A. A. Albadr, S. Tiun, M. Ayob, and F. T. AL-Dhief, “Spoken language identification based on optimised genetic algorithm–extreme learning machine approach,” Int. J. Speech Technol., vol. 22, no. 3, pp. 711–727, 2019, doi: 10.1007/s10772-019-09621-w.
[13] X. Xiao, M. Yan, S. Basodi, C. Ji, and Y. Pan, “Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm,” 2020. doi : 10.48550/arXiv.2006.12703.
[14] Y. Sun, B. Xue, M. Zhang, G. G. Yen, and J. Lv, “Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification,” IEEE Trans. Cybern., vol. 50, no. 9, pp. 3840–3854, 2020, doi: 10.1109/TCYB.2020.2983860.
[15] P. Nirwan and G. Singh, “Segmentation and identification of bilingual offline handwritten scripts (devanagari and roman),” Int. J. Recent Technol. Eng., vol. 8, no. 2 Special Issue 6, pp. 603–607, 2019, doi: 10.35940/ijrte.B1178.0782S619.
[16] A. J. Reiling, “Convolutional Neural Network Optimization Using Genetic Algorithms,” 2017. Available at : Semantic Scholar.
[17] A. Bhandare and D. Kaur, “Designing convolutional neural network architecture using genetic algorithms,” in 2018 World Congress in Computer Science, Computer Engineering and Applied Computing, CSCE 2018 - Proceedings of the 2018 International Conference on Artificial Intelligence, ICAI 2018, 2018, pp. 150–156. doi: 10.21307/ijanmc-2021-024.
[18] F. Mattioli, D. Caetano, A. Cardoso, E. Naves, and E. Lamounier, “An experiment on the use of genetic algorithms for topology selection in deep learning,” J. Electr. Comput. Eng., vol. 2019, 2019, doi: 10.1155/2019/3217542.
[19] S. Loussaief and A. Abdelkrim, “Convolutional Neural Network hyper-parameters optimization based on Genetic Algorithms,” Int. J. Adv. Comput. Sci. Appl., vol. 9, no. 10, pp. 252–266, 2018, doi: 10.14569/IJACSA.2018.091031.
[20] A. Hassanat, K. Almohammadi, E. Alkafaween, E. Abunawas, A. Hammouri, and V. B. S. Prasath, “Choosing mutation and crossover ratios for genetic algorithms-a review with a new dynamic approach,” Inf., vol. 10, no. 12, 2019, doi: 10.3390/info10120390.
[21] F. Johnson, A. Valderrama, C. Valle, B. Crawford, R. Soto, and R. Nanculef, “Automating Configuration of Convolutional Neural Network Hyperparameters Using Genetic Algorithm,” IEEE Access, vol. 8, pp. 156139–156152, 2020, doi: 10.1109/ACCESS.2020.3019245.
[22] C. Li et al., “Genetic algorithm based hyper-parameters optimization for transfer convolutional neural network,” 2022. doi: 10.1117/12.2637170.
[23] J. H. Yoo, H. Il Yoon, H. G. Kim, H. S. Yoon, and S. S. Han, “Optimization of Hyper-parameter for CNN Model using Genetic Algorithm,” 2019 IEEE Int. Conf. Electr. Control Instrum. Eng. ICECIE 2019 - Proc., 2019, doi: 10.1109/ICECIE47765.2019.8974762.
[24] R. Zatarain Cabada, H. Rodriguez Rangel, M. L. Barron Estrada, and H. M. Cardenas Lopez, “Hyperparameter optimization in CNN for learning-centered emotion recognition for intelligent tutoring systems,” Soft Comput., vol. 24, no. 10, pp. 7593–7602, 2020, doi: 10.1007/s00500-019-04387-4.
[25] A. Shrestha and A. Mahmood, “Optimizing deep neural network architecture with enhanced genetic algorithm,” in Proceedings - 18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019, 2019, pp. 1365–1370. doi: 10.1109/ICMLA.2019.00222.
[26] A. Baldominos, Y. Saez, and P. Isasi, “Model Selection in Committees of Evolved Convolutional Neural Networks Using Genetic Algorithms,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 11314 LNCS, pp. 364–373, 2018, doi: 10.1007/978-3-030-03493-1_39.
[27] P. J. Grother and K. K. Hanaoka, “NIST Special Database 19,” 2016. doi : 10.18434/T4H01C.
[28] N. K. Manaswi, “Deep Learning with Applications Using Python,” Deep Learning with Applications Using Python. 2018. doi: 10.1007/978-1-4842-3516-4.
[29] S. Katoch, S. S. Chauhan, and V. Kumar, “A review on genetic algorithm: past, present, and future,” Multimed. Tools Appl., vol. 80, no. 5, pp. 8091–8126, 2021, doi: 10.1007/s11042-020-10139-6.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0