Citation: | ZHANG Hong, JIANG Xiaogang, ZHU Zhiwei, XIA Runchuan, ZHOU Jianting. Review on Intelligent Image Recognition of Apparent Diseases of Stay Cable[J]. Journal of Southwest Jiaotong University. doi: 10.3969/j.issn.0258-2724.20220647 |
The stay cable is one of the main load-bearing elements of cable-stayed bridges, and the disease of its outer sheath is easy to penetrate inside the cable and affect the health of the steel wire. Therefore, it is significant to use the video image method to intelligently identify the apparent disease of the cable. Based on image recognition, the methods of apparent disease recognition for stay cable were systematically reviewed from two aspects: traditional image detection and deep learning. The basic principles and application effects of each method were introduced, and the current detection examples were analyzed. Some cutting-edge deep learning methods were introduced to provide a reference for the apparent detection of cables. The main features of various methods were summarized, and the problems existing in the current detection were discussed and prospected. The deep learning model-based image recognition method had better recognition accuracy and algorithm robustness, stronger learning ability and adaptability, and optimal comprehensive image defect recognition effect, but there were still difficulties such as the difficult balance between detection accuracy and speed, large image data demand, and high labeling cost. To this end, detection methods could be improved by improving image quality, constructing more semi-supervised and unsupervised deep learning models, and enhancing the learning ability of detection models.
[1] |
《中国公路学报》编辑部. 中国桥梁工程学术研究综述•2021[J]. 中国公路学报,2021,34(2): 1-97. doi: 10.3969/j.issn.1001-7372.2021.02.002
Editorial Department of China Journal of Highway and Transport. Review on China’s bridge engineering research: 2021[J]. China Journal of Highway and Transport, 2021, 34(2): 1-97. doi: 10.3969/j.issn.1001-7372.2021.02.002
|
[2] |
苑仁安,张明金,郑清刚,等. 超大跨斜拉桥横桥向恒载非对称力学行为[J]. 西南交通大学学报,2023,58(3): 527-534. doi: 10.3969/j.issn.0258-2724.20210279
YUAN Renan, ZHANG Mingjin, ZHENG Qinggang, et al. Mechanical characteristics of super-long-span cable-stayed bridge with transverse asymmetrical load[J]. Journal of Southwest Jiaotong University, 2023, 58(3): 527-534. doi: 10.3969/j.issn.0258-2724.20210279
|
[3] |
LIU S W, SUN Y H, JIANG X Y, et al. A review of wire rope detection methods, sensors and signal processing techniques[J]. Journal of Nondestructive Evaluation, 2020, 39: 1-18. doi: 10.1007/s10921-019-0643-0
|
[4] |
钱骥,孙利民,蒋永. 桥梁拉索损伤声发射监测研究进展[J]. 应用声学,2016,35(4): 369-376.
QIAN Ji, SUN Limin, JIANG Yong. Advances in acoustic emission technique to cables health monitoring[J]. Journal of Applied Acoustics, 2016, 35(4): 369-376.
|
[5] |
中华人民共和国交通运输部. 在用公路桥梁现场检测技术规程:JTG/T 5214—2022[S]. 北京:人民交通出版社,2022.
|
[6] |
刘倩倩,徐程,韩勇涛,等. 一种用于斜拉桥拉索套管的喷涂装置:CN211051767U[P]. 2020-07-21.
|
[7] |
WANG Z P, HE B, ZHOU Y M, et al. Design and implementation of a cable inspection robot for cable-stayed bridges[J]. Robotica, 2021: 1-17.
|
[8] |
LI J, YIN C Y, SHI Y D, et al. Circumferentially rotatable inspection robot with elastic suspensions for bridge cables[J]. Industrial Robot: the International Journal of Robotics Research and Application, 2022, 49(5): 981-993. doi: 10.1108/IR-11-2021-0261
|
[9] |
马晔,邹露鹏,张理轻. 无人机加载光学摄像及红外成像系统对海上特大桥塔索质量检测的运用技术[J]. 公路交通科技,2018,35(8): 89-93,105.
MA Ye, ZOU Lupeng, ZHANG Liqing. Application technology of UAV equipped with optical camera and digital infrared imagery in inspecting quality of pylon and stay cable of sea-crossing bridge[J]. Journal of Highway and Transportation Research and Development, 2018, 35(8): 89-93,105.
|
[10] |
杨传礼,张修庆. 基于机器视觉和深度学习的材料缺陷检测应用综述[J]. 材料导报,2022,36(16): 226-234.
YANG Chuanli, ZHANG Xiuqing. Survey of applications of material defect detection based on machine vision and deep learning[J]. Materials Reports, 2022, 36(16): 226-234.
|
[11] |
赵朗月,吴一全. 基于机器视觉的表面缺陷检测方法研究进展[J]. 仪器仪表学报,2022,43(1): 198-219.
ZHAO Langyue, WU Yiquan. Research progress of surface defect detection methods based on machine vision[J]. Chinese Journal of Scientific Instrument, 2022, 43(1): 198-219.
|
[12] |
黄梦涛,连一鑫. 基于改进Canny算子的锂电池极片表面缺陷检测[J]. 仪器仪表学报,2021,42(10): 199-209.
HUANG Mengtao, LIAN Yixin. Lithium battery electrode plate surface defect detection based on improved Canny operator[J]. Chinese Journal of Scientific Instrument, 2021, 42(10): 199-209.
|
[13] |
DHIVYA, M, RENUKA D M. Detection of structural defects in fabric parts using a novel edge detection method[J]. The Computer Journal, 2019, 62(7): 1036-1043. doi: 10.1093/comjnl/bxy121
|
[14] |
WANG S, LIU X Q, YANG T F, et al. Panoramic crack detection for steel beam based on structured random forests[J]. IEEE Access, 2018, 6: 16432-16444. doi: 10.1109/ACCESS.2018.2812141
|
[15] |
SINGH, S A, DESAI K A. Automated surface defect detection framework using machine vision and convolutional neural networks[J]. Journal of Intelligent Manufacturing, 2023, 34(4): 1995-2011. doi: 10.1007/s10845-021-01878-w
|
[16] |
郭继昌,岳惠惠,张怡,等. 图像增强对显著性目标检测的影响研究[J]. 中国图象图形学报,2022,27(7): 2129-2147. doi: 10.11834/j.issn.1006-8961.2022.7.zgtxtxxb-a202207005
GUO Jichang, YUE Huihui, ZHANG Yi, et al. The analysis of image enhancement on salient object detection[J]. Journal of Image and Graphics, 2022, 27(7): 2129-2147. doi: 10.11834/j.issn.1006-8961.2022.7.zgtxtxxb-a202207005
|
[17] |
王田,邹子龙,乔美娜. 基于图像特征分析的物体轮廓提取[J]. 北京航空航天大学学报,2016,42(8): 1762-1768.
WANG Tian, ZOU Zilong, QIAO Meina. Object contour extraction based on image feature analysis[J]. Journal of Beijing University of Aeronautics and Astronautics, 2016, 42(8): 1762-1768.
|
[18] |
刘朝涛,杜子学,武维,等. 基于图像处理的桥梁缆索检测系统研制[J]. 重庆交通大学学报(自然科学版),2014,33(2): 5-9.
LIU Chaotao, DU Zixue, WU Wei, et al. Development of cable inspection system based on image processing[J]. Journal of Chongqing Jiaotong University (Natural Science), 2014, 33(2): 5-9.
|
[19] |
蔡步文. 基于图像处理的斜拉桥缆索表面缺陷检测的研究[D]. 重庆:重庆大学,2010.
|
[20] |
王建林,潘建平. 一种桥梁拉索表面缺陷图像的背景分割方法研究[J]. 测绘工程,2018,27(7): 24-29.
WANG Jianlin, PAN Jianping. A method of background segmentation for surface defect of bridge cables[J]. Engineering of Surveying and Mapping, 2018, 27(7): 24-29.
|
[21] |
阳振宇. 基于MATLAB的桥梁拉索表面缺陷视频检测[D]. 重庆:重庆交通大学,2019.
|
[22] |
CHEN J N, WANG H, TU C L, et al. Surface defect detection of cable based on threshold image difference[C]//2021 IEEE Far East NDT New Technology & Application Forum (FENDT). Kunming: IEEE, 2022: 185-190.
|
[23] |
高潮,任可,郭永彩,等. 基于DSP和图像识别的拉索表面缺陷检测技术[J]. 重庆大学学报(自然科学版),2007,30(9): 36-38.
GAO Chao, REN Ke, GUO Yongcai, et al. Detecting technology of cabel surface defect based on DSP and image recognition[J]. Journal of Chongqing University (Natural Science Edition), 2007, 30(9): 36-38.
|
[24] |
李新科,高潮,郭永彩,等. 桥梁拉索表面缺陷的分布式机器视觉检测[J]. 光学技术,2013,39(5): 424-428.
LI Xinke, GAO Chao, GUO Yongcai, et al. The inspection method based on distributed machine vision for surface defects of bridge cables[J]. Optical Technique, 2013, 39(5): 424-428.
|
[25] |
刘淑敏. 基于机器视觉的桥梁裂纹检测算法研究[D]. 重庆:重庆大学,2019.
|
[26] |
LIU X M, TIAN H, WANG Y, et al. Research on image segmentation algorithm and performance of power insulator based on adaptive region growing[J]. Journal of Electrical Engineering & Technology, 2022, 17(6): 3601-3612.
|
[27] |
蔡步文,周忆,申超. 斜拉桥缆索表面缺陷提取方法研究[J]. 微计算机信息,2010,26(32): 188-190.
CAI Buwen, ZHOU Yi, SHEN Chao. Crack detection algorithm design for cable’s protective layer surface on cable-stayed bridge[J]. Microcomputer Information, 2010, 26(32): 188-190.
|
[28] |
LI L Y, HAO P F. Steel plate corrugation defect intelligent detection method based on picture cropping and region growing algorithm[C]//2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA). Xi’an: IEEE, 2019: 587-590.
|
[29] |
ZOU G F, LI T T, LI G Y, et al. A visual detection method of tile surface defects based on spatial-frequency domain image enhancement and region growing[C]//2019 Chinese Automation Congress (CAC). Hangzhou: IEEE, 2020: 1631-1636.
|
[30] |
HO H N, KIM K D, PARK Y S, et al. An efficient image-based damage detection for cable surface in cable-stayed bridges[J]. NDT & E International, 2013, 58: 18-23.
|
[31] |
周兆逸,张亚南,王肖锋,等. 基于改正二维主成分分析的焊缝表面缺陷检测[J]. 焊接学报,2021,42(11): 70-76,101.
ZHOU Zhaoyi, ZHANG Yanan, WANG Xiaofeng, et al. Weld surface defect detection based on improved twodimensional principal component analysis[J]. Transactions of the China Welding Institution, 2021, 42(11): 70-76,101.
|
[32] |
王欣,高炜欣,王征,等. 基于并行计算的PCA在缺陷检测中的应用[J]. 计算机工程与设计,2016,37(10): 2810-2815,2850.
WANG Xin, GAO Weixin, WANG Zheng, et al. Application of PCA in defect detection based on parallel computing[J]. Computer Engineering and Design, 2016, 37(10): 2810-2815,2850.
|
[33] |
ZHOU Y H, MA Z X, SHI X W, et al. An adaptive clustering method detecting the surface defects on linear guide rails[J]. International Journal of Computer Integrated Manufacturing, 2019, 32(8): 798-808. doi: 10.1080/0951192X.2019.1636409
|
[34] |
LI X G, ZHU J A, SHI H R, et al. Surface defect detection of seals based on K-means clustering algorithm and particle swarm optimization[J]. Scientific Programming, 2021(1): 3965247.1-3965247.12.
|
[35] |
YANG H, CHEN Y F, SONG K Y, et al. Multiscale feature-clustering-based fully convolutional autoencoder for fast accurate visual inspection of texture surface defects[J]. IEEE Transactions on Automation Science and Engineering, 2019, 16(3): 1450-1467. doi: 10.1109/TASE.2018.2886031
|
[36] |
LI X K, GUO Y C, LI Y M. Particle swarm optimization-based SVM for classification of cable surface defects of the cable-stayed bridges[J]. IEEE Access, 2019, 8: 44485-44492.
|
[37] |
李鑫. 桥梁拉索护套表面缺陷视觉检测及智能识别研究[D]. 长沙:湖南大学,2019.
|
[38] |
ZHANG K B, YAN Y D, LI P F, et al. Fabric defect detection using salience metric for color dissimilarity and positional aggregation[J]. IEEE Access, 2018, 6: 49170-49181. doi: 10.1109/ACCESS.2018.2868059
|
[39] |
CHANG Z Y, CAO J, ZHANG Y Z. A novel image segmentation approach for wood plate surface defect classification through convex optimization[J]. Journal of Forestry Research, 2018, 29(6): 1789-1795. doi: 10.1007/s11676-017-0572-7
|
[40] |
JAWAHAR M, CHANDRA BABU N K, VANI K, et al. Vision based inspection system for leather surface defect detection using fast convergence particle swarm optimization ensemble classifier approach[J]. Multimedia Tools and Applications, 2021, 80(3): 4203-4235. doi: 10.1007/s11042-020-09727-3
|
[41] |
夏毅敏,李清友,邓朝辉,等. 基于轻量级模型的隧道岩性快速识别方法[J]. 西南交通大学学报,2021,56(2): 420-427.
XIA Yimin, LI Qingyou, DENG Chaohui, et al. Rapid identification method for lithology of tunnel based on lightweight model[J]. Journal of Southwest Jiaotong University, 2021, 56(2): 420-427.
|
[42] |
范新南,汪杰,史朋飞,等. 基于头脑风暴优化的PCNN路面裂缝分割算法[J]. 西南交通大学学报,2021,56(3): 572-578.
FAN Xinnan, WANG Jie, SHI Pengfei, et al. Pavement crack segmentation algorithm based on pulse coupled neural network with brainstorming optimization[J]. Journal of Southwest Jiaotong University, 2021, 56(3): 572-578.
|
[43] |
THAKUR M, PILLAI S K. A review on various methods for classification of massive noisy image[C]//2019 International Conference on Intelligent Sustainable Systems (ICISS). Palladam: IEEE, 2019: 525-528.
|
[44] |
HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
|
[45] |
董斌. 斜拉桥拉索外套表观病害与索力时变识别方法研究[D]. 南京:东南大学,2020.
|
[46] |
JIANG X Z, WANG X F, CHEN D F. Research on defect detection of castings based on deep residual network[C]//2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI). Beijing: IEEE, 2019: 1-6.
|
[47] |
CHEN X Y, WANG D Y, SHAO J J, et al. Plastic gasket defect detection based on transfer learning[J]. Scientific Programming, 2021(1): 5990020.1-5990020.11.
|
[48] |
王耀东,朱力强,余祖俊,等. 基于样本自动标注的隧道裂缝病害智能识别研究[J]. 西南交通大学学报,2023,58(5):1-8.
WANG Yaodong, ZHU Liqiang, YU Zhujun, et al. Research of tunnel cracks recognition based on automatic sample labeling[J]. Journal of Southwest Jiaotong University,2023,58(5):1-8.
|
[49] |
REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
|
[50] |
余朝阳. 斜拉索检测机器人的智能表观检测研究[J]. 公路交通技术,2020,36(2): 86-92.
YU Chaoyang. Research on intelligent apperance algorithm of stay cable detection robot[J]. Technology of Highway and Transport, 2020, 36(2): 86-92.
|
[51] |
晏班夫,徐观亚,栾健,等. 基于Faster R-CNN与形态法的路面病害识别[J]. 中国公路学报,2021,34(9): 181-193. doi: 10.3969/j.issn.1001-7372.2021.09.015
YAN Banfu, XU Guanya, LUAN Jian, et al. Pavement distress detection based on faster R-CNN and morphological operations[J]. China Journal of Highway and Transport, 2021, 34(9): 181-193. doi: 10.3969/j.issn.1001-7372.2021.09.015
|
[52] |
李东洁,李若昊. 基于改进Faster RCNN的马克杯缺陷检测方法[J]. 激光与光电子学进展,2020,57(4): 353-360.
LI Dongjie, LI Ruohao. Mug defect detection method based on improved faster RCNN[J]. Laser & Optoelectronics Progress, 2020, 57(4): 353-360.
|
[53] |
LI L, JIANG Z J, LI Y N. Surface defect detection algorithm of aluminum based on improved faster RCNN[C]//2021 IEEE 9th International Conference on Information, Communication and Networks (ICICN). Xi’an: IEEE, 2022: 527-531.
|
[54] |
REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 779-788.
|
[55] |
李运堂,谢梦鸣,王鹏峰,等. 基于改进YOLOV3算法的斜拉桥拉索表面缺陷检测方法[J]. 传感技术学报,2021,34(11): 1509-1517. doi: 10.3969/j.issn.1004-1699.2021.11.014
LI Yuntang, XIE Mengming, WANG Pengfeng, et al. Defects inspection method for cable surface of cable-stayed bridge based on improved YOLOV3 algorithm[J]. Chinese Journal of Sensors and Actuators, 2021, 34(11): 1509-1517. doi: 10.3969/j.issn.1004-1699.2021.11.014
|
[56] |
张可惠. 桥梁斜拉索表观缺陷视觉检测系统研究[D]. 长沙:湖南大学,2021.
|
[57] |
周朗明,万智,胡帅花,等. 一种斜拉桥缆索表面多类型病害智能检测和识别方法:CN113313107A[P]. 2021-08-27.
|
[58] |
LI J Y, SU Z F, GENG J H, et al. Real-time detection of steel strip surface defects based on improved YOLO detection network[J]. IFAC-PapersOnLine, 2018, 51(21): 76-81. doi: 10.1016/j.ifacol.2018.09.412
|
[59] |
CHEN Y B, FU Q S, WANG G T. Surface defect detection of nonburr cylinder liner based on improved YOLOv4[J]. Mobile Information Systems, 2021, 2021: 1-13.
|
[60] |
RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
|
[61] |
QIAN K. Automated detection of steel defects via machine learning based on real-time semantic segmentation[C]//Proceedings of the 3rd International Conference on Video and Image Processing. Shanghai: ACM, 2019: 42-46.
|
[62] |
李忠海,白秋阳,王富明,等. 基于语义分割的钢轨表面缺陷实时检测系统[J]. 计算机工程与应用,2021,57(12): 248-256. doi: 10.3778/j.issn.1002-8331.2004-0101
LI Zhonghai, BAI Qiuyang, WANG Fuming, et al. Real-time detection system of rail surface defects based on semantic segmentation[J]. Computer Engineering and Applications, 2021, 57(12): 248-256. doi: 10.3778/j.issn.1002-8331.2004-0101
|
[63] |
LIANG Z C, ZHANG H, LIU L, et al. Defect detection of rail surface with deep convolutional neural networks[C]//2018 13th World Congress on Intelligent Control and Automation (WCICA). Changsha: IEEE, 2019: 1317-1322.
|
[64] |
TANG C W, FENG X X, WEN H T, et al. Semantic segmentation network for surface defect detection of automobile wheel hub fusing high-resolution feature and multi-scale feature[J]. Applied Sciences, 2021, 11(22): 10508. doi: 10.3390/app112210508
|
[65] |
HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 2980-2988.
|
[66] |
王兴松,李杰,田梦倩. 基于深度学习的桥梁拉索表面缺陷实时识别系统及方法:CN112101138A[P]. 2020-12-18.
|
[67] |
HOU S T, DONG B, WANG H C, et al. Inspection of surface defects on stay cables using a robot and transfer learning[J]. Automation in Construction, 2020, 119: 103382. doi: 10.1016/j.autcon.2020.103382
|
[68] |
余加勇,李锋,薛现凯,等. 基于无人机及Mask R-CNN的桥梁结构裂缝智能识别[J]. 中国公路学报,2021,34(12): 80-90. doi: 10.3969/j.issn.1001-7372.2021.12.007
YU Jiayong, LI Feng, XUE Xiankai, et al. Intelligent identification of bridge structural cracks based on unmanned aerial vehicle and mask R-CNN[J]. China Journal of Highway and Transport, 2021, 34(12): 80-90. doi: 10.3969/j.issn.1001-7372.2021.12.007
|
[69] |
CHEN W Y, TSAO Y R, LAI J Y, et al. Real-time instance segmentation of metal screw defects based on deep learning approach[J]. Measurement Science Review, 2022, 22(3): 107-111. doi: 10.2478/msr-2022-0014
|
[70] |
ZHANG G W, PAN Y, ZHANG L M. Semi-supervised learning with GAN for automatic defect detection from images[J]. Automation in Construction, 2021, 128: 103764.1-103764.14. doi: 10.1016/j.autcon.2021.103764
|
[71] |
ZHAO Z X, LI B, DONG R, et al. A surface defect detection method based on positive samples[C]//Pacific Rim International Conference on Artificial Intelligence. Cham: Springer, 2018: 473-481.
|
[72] |
PENG Z R, GONG X Y, WEI B G, et al. Automatic unsupervised fabric defect detection based on self-feature comparison[J]. Electronics, 2021, 10(21): 2652.1-2652.12. doi: 10.3390/electronics10212652
|
[73] |
MATUSZCZYK, D, TSCHORN N, WEICHERT F. Deep learning based synthetic image generation for defect detection in additive manufacturing industrial environments[C]//2022 7th International Conference on Mechanical Engineering and Robotics Research (ICMERR). Krakow: IEEE, 2022: 209-218.
|
[74] |
YUN J P, SHIN W C, KOO G, et al. Automated defect inspection system for metal surfaces based on deep learning and data augmentation[J]. Journal of Manufacturing Systems, 2020, 55: 317-324. doi: 10.1016/j.jmsy.2020.03.009
|