(Munzur University, Turkey)(2) Mehmet Karakose
(Firat University, Turkey)*corresponding author
AbstractThe railways, which are frequently used by countries for both passenger and freight transportation, should be checked periodically. Controls made with classical methods are slow and there are often overlooked faults. This work suggests a novel deep learning-based technique for identifying fastener and railway track surface defects. Within the scope of the proposed method, firstly, The railroad track was observed using an autonomous drone. Shaky images in the recorded video were removed with a video stabilization algorithm. Frames were created and labeled from the video and rail and fastener regions were detected using the Faster R-CNN deep neural network. Fault detection was performed through ResNet101v2-based classification using different datasets for identifying surface detects in rails and different datasets for detection of fasteners. The proposed method was experimentally shown to have a 98% accuracy rate for detecting rail surface flaws and a 95% accuracy rate for detecting fastener flaws. An user interface was developed to display the identified faulty images on computers, tablets and mobile phones via a mobile application. The system, which was previously proposed in a different study, was retrained by going through the video stabilization step, thus improving the fault detection rate, and the method was also included in the user interface module. This study contributes to the processing of ever-increasing video data with deep learning-based methods. It is also anticipated that it will benefit researchers working in the field of railway non-contact fault detection.
KeywordsDefect detection Deep learning Autonomous drone Faster R-CNN ResNet101v2
|
DOIhttps://doi.org/10.26555/ijain.v10i2.1237 |
Article metricsAbstract views : 220 | PDF views : 48 |
Cite |
Full Text Download
|
References
[1] P. Bojarczak and P. Lesiak, “UAVs in rail damage image diagnostics supported by deep-learning networks,” Open Eng., vol. 11, no. 1, pp. 339–348, Jan. 2021, doi: 10.1515/eng-2021-0033.
[2] M. Yilmazer and M. Karakose, “Mask R-CNN Architecture Based Railway Fastener Fault Detection Approach,” in 2022 International Conference on Decision Aid Sciences and Applications (DASA), Mar. 2022, pp. 1363–1366, doi: 10.1109/DASA54658.2022.9765024.
[3] M. Yilmazer and M. Karakose, “Railway Condition Monitoring and Fault Detection Based on YOLOv4,” in 2022 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT), Nov. 2022, pp. 439–443, doi: 10.1109/3ICT56508.2022.9990640.
[4] M.-Y. Cheng, R. R. Khasani, and K. Setiono, “Image quality enhancement using HybridGAN for automated railway track defect recognition,” Autom. Constr., vol. 146, p. 104669, Feb. 2023, doi: 10.1016/j.autcon.2022.104669.
[5] M. G. Minguell and R. Pandit, “TrackSafe: A comparative study of data-driven techniques for automated railway track fault detection using image datasets,” Eng. Appl. Artif. Intell., vol. 125, p. 106622, Oct. 2023, doi: 10.1016/j.engappai.2023.106622.
[6] N. Mukhtiar, K. Khuwaja, D. P. Tocut, and I. Tarca, “An Experimental Study on the Railway Track Surface Fault Detection with Automation,” in 2023 17th International Conference on Engineering of Modern Electric Systems (EMES), Jun. 2023, pp. 1–4, doi: 10.1109/EMES58375.2023.10171771.
[7] H. Luo, L. Cai, and C. Li, “Rail Surface Defect Detection Based on An Improved YOLOv5s,” Appl. Sci., vol. 13, no. 12, p. 7330, Jun. 2023, doi: 10.3390/app13127330.
[8] A. K. Singh, A. K. Dwivedi, N. Nahar, and D. Singh, “Railway Track Sleeper Detection in Low Altitude UAV Imagery Using Deep Convolutional Neural Network,” in 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Jul. 2021, pp. 355–358, doi: 10.1109/IGARSS47720.2021.9554542.
[9] S. Qiu et al., “Automated detection of railway defective fasteners based on YOLOv8-FAM and synthetic data using style transfer,” Autom. Constr., vol. 162, p. 105363, Jun. 2024, doi: 10.1016/j.autcon.2024.105363.
[10] J. Liu, H. Liu, C. Chakraborty, K. Yu, X. Shao, and Z. Ma, “Cascade Learning Embedded Vision Inspection of Rail Fastener by Using a Fault Detection IoT Vehicle,” IEEE Internet Things J., vol. 10, no. 4, pp. 3006–3017, Feb. 2023, doi: 10.1109/JIOT.2021.3126875.
[11] P. Chandran, J. Asber, F. Thiery, J. Odelius, and M. Rantatalo, “An Investigation of Railway Fastener Detection Using Image Processing and Augmented Deep Learning,” Sustainability, vol. 13, no. 21, p. 12051, Oct. 2021, doi: 10.3390/su132112051.
[12] M. Chen, W. Zhai, S. Zhu, L. Xu, and Y. Sun, “Vibration-based damage detection of rail fastener using fully convolutional networks,” Veh. Syst. Dyn., vol. 60, no. 7, pp. 2191–2210, Jul. 2022, doi: 10.1080/00423114.2021.1896010.
[13] L. Wang, Q. Zang, K. Zhang, and L. Wu, “A rail fastener defect detection algorithm based on improved YOLOv5,” Feb. 2024, doi: 10.1177/09544097241234380.
[14] Z. Chen, Q. Wang, Q. He, T. Yu, M. Zhang, and P. Wang, “CUFuse: Camera and Ultrasound Data Fusion for Rail Defect Detection,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 11, pp. 21971–21983, Nov. 2022, doi: 10.1109/TITS.2022.3189677.
[15] X. Li, Q. Wang, X. Yang, K. Wang, and H. Zhang, “Track Fastener Defect Detection Model Based on Improved YOLOv5s,” Sensors, vol. 23, no. 14, p. 6457, Jul. 2023, doi: 10.3390/s23146457.
[16] R. Shafique et al., “A Novel Approach to Railway Track Faults Detection Using Acoustic Analysis,” Sensors, vol. 21, no. 18, p. 6221, Sep. 2021, doi: 10.3390/s21186221.
[17] İ. Aydin, M. Sevi, E. Akin, E. Güçlü, M. Karaköse, and H. Aldarwich, “A Deep Learning-Based Hybrid Approach to Detect Fastener Defects in Real-Time,” Teh. Vjesn. - Tech. Gaz., vol. 30, no. 5, pp. 1461–1468, Oct. 2023, doi: 10.17559/TV-20221020152721.
[18] D. Zheng et al., “A Defect Detection Method for Rail Surface and Fasteners Based on Deep Convolutional Neural Network,” Comput. Intell. Neurosci., vol. 2021, pp. 1–15, Jul. 2021, doi: 10.1155/2021/2565500.
[19] Q. Zhou, “A Detection System for Rail Defects Based on Machine Vision,” J. Phys. Conf. Ser., vol. 1748, no. 2, p. 022012, Jan. 2021, doi: 10.1088/1742-6596/1748/2/022012.
[20] Y. Wu, Y. Qin, Y. Qian, F. Guo, Z. Wang, and L. Jia, “Hybrid deep learning architecture for rail surface segmentation and surface defect detection,” Comput. Civ. Infrastruct. Eng., vol. 37, no. 2, pp. 227–244, Feb. 2022, doi: 10.1111/mice.12710.
[21] Y. Zheng, S. Wu, D. Liu, R. Wei, S. Li, and Z. Tu, “Sleeper Defect Detection Based on Improved YOLO V3 Algorithm,” in 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA), Nov. 2020, pp. 955–960, doi: 10.1109/ICIEA48937.2020.9248299.
[22] J. Hu et al., “High Speed Railway Fastener Defect Detection by Using Improved YoLoX-Nano Model,” Sensors, vol. 22, no. 21, p. 8399, Nov. 2022, doi: 10.3390/s22218399.
[23] Z. Su, K. Han, W. Song, and K. Ning, “Railway fastener defect detection based on improved YOLOv5 algorithm,” in 2022 IEEE 6th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC ), Oct. 2022, vol. 2022-Octob, pp. 1923–1927, doi: 10.1109/IAEAC54830.2022.9929911.
[24] R. Kapoor, R. Goel, and A. Sharma, “An intelligent railway surveillance framework based on recognition of object and railway track using deep learning,” Multimed. Tools Appl., vol. 81, no. 15, pp. 21083–21109, Jun. 2022, doi: 10.1007/s11042-022-12059-z.
[25] Y. Qiu, H. Liu, J. Liu, and B. Shi, “Center-Triplet Loss for Railway Defective Fastener Detection,” IEEE Sens. J., vol. 24, no. 3, pp. 3180–3190, Feb. 2024, doi: 10.1109/JSEN.2023.3339883.
[26] M. A.-M. Khan, S.-H. Kee, and A.-A. Nahid, “Vision-Based Concrete-Crack Detection on Railway Sleepers Using Dense U-Net Model,” Algorithms, vol. 16, no. 12, p. 568, Dec. 2023, doi: 10.3390/a16120568.
[27] Y. Santur, M. Yilmazer, M. Karakose, and E. Akin, “A New Rail Surface Defects Detection Approach Using 3D Laser Cameras Based on ResNet50,” Trait. du Signal, vol. 39, no. 4, pp. 1339–1345, Aug. 2022, doi: 10.18280/ts.390427.
[28] A. Saini, D. Singh, and M. Alvarez, “FishTwoMask R-CNN: Two-stage Mask R-CNN approach for detection of fishplates in high-altitude railroad track drone images,” Multimed. Tools Appl., vol. 83, no. 4, pp. 10367–10392, Jan. 2024, doi: 10.1007/s11042-023-15924-7.
[29] T. Bai, J. Gao, J. Yang, and D. Yao, “A Study on Railway Surface Defects Detection Based on Machine Vision,” Entropy, vol. 23, no. 11, p. 1437, Oct. 2021, doi: 10.3390/e23111437.
[30] I. Aydin, E. Akin, and M. Karakose, “Defect classification based on deep features for railway tracks in sustainable transportation,” Appl. Soft Comput., vol. 111, p. 107706, Nov. 2021, doi: 10.1016/j.asoc.2021.107706.
[31] M. Yilmazer, M. Karaköse, İ. Aydin, and E. Akin, “Mask R-CNN Derin Sinir Ağı ile Demiryolu Hattı Bileşenlerinde Çoklu Hata Tespiti,” Çukurova Üniversitesi Mühendislik Fakültesi Derg., vol. 37, no. 4, pp. 1103–1112, Dec. 2022, doi: 10.21605/cukurovaumfd.1230955.
[32] R. Girshick, “Fast R-CNN,” in 2015 IEEE International Conference on Computer Vision (ICCV), Dec. 2015, vol. 2015 Inter, pp. 1440–1448, doi: 10.1109/ICCV.2015.169.
[33] K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in 2017 IEEE International Conference on Computer Vision (ICCV), Oct. 2017, vol. 2017-Octob, pp. 2980–2988, doi: 10.1109/ICCV.2017.322.
[34] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-December, pp. 770–778, Dec. 2016, doi: 10.1109/CVPR.2016.90.
[35] X. Giben, V. M. Patel, and R. Chellappa, “Material classification and semantic segmentation of railway track images with deep convolutional neural networks,” in 2015 IEEE International Conference on Image Processing (ICIP), Sep. 2015, vol. 2015-Decem, pp. 621–625, doi: 10.1109/ICIP.2015.7350873.
[36] F. Guo, Y. Qian, and Y. Shi, “Real-time railroad track components inspection based on the improved YOLOv4 framework,” Autom. Constr., vol. 125, p. 103596, May 2021, doi: 10.1016/j.autcon.2021.103596.
[37] Y. Min, J. Guo, and K. Yang, “Research On Real-time Detection Algorithm Of Rail-surface Defects Based On Improved YOLOX,” J. Appl. Sci. Eng., vol. 26, no. 6, pp. 799–810, 2022. [Online]. Available at: http://jase.tku.edu.tw/articles/jase-202306-26-6-0006.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0