Visual Localization Method Research of Fatigue Crack in Steel Box Girder Based on Registration Assisted by Weld
-
摘要:
为解决钢箱梁复杂构造下疲劳裂纹定位难题，利用焊缝的线性几何特征提出一种裂纹-结构配准算法，以实现裂纹“识别、量化、定位”感知全过程的集成与自动化. 首先，构建一种双任务深度神经网络CrackFormer-Hough，在二维图像中同时识别裂纹与焊缝；然后，利用立体视觉技术对识别目标进行三维重建，生成目标点云并量化其几何特征；最后，以焊缝为配准参照物，基于粗/精点云配准算法获取相机与结构空间的坐标变换关系，依据结构坐标将各分镜下的裂纹片段进行拼接融合，并在结构地图中绘制出裂纹完整形态. 现场测试结果表明：通过引入深度霍夫变换来学习钢箱梁场景语义，识别网络能够克服背景干扰准确辨别特定部位的焊缝构造；借助焊缝显著的线性特征，所提配准算法仅需稀疏点云便可将配准误差控制在1 mm左右；疲劳裂纹感知表现并不敏感于相机姿态，不同镜头下平均定位误差在5 mm以内，长度测量误差仅为1%左右.
Abstract:To address the localization problem of fatigue cracks in steel box girders with complex structures, a registration algorithm for crack and structure was proposed by leveraging the linear geometric features of welds. Consequently, the whole perception process for identifying, quantifying, and localizing the cracks was integrated and automated. Firstly, a dual-task deep neural network, CrackFormer-Hough, was established to identify the cracks and welds in two-dimensional (2D) images simultaneously. Then, the identified targets were reconstructed by stereo vision techniques to generate target point clouds and quantify their geometric features. Finally, by taking welds as the reference, the coordinate transformation relationship between the camera and structure space was obtained based on the coarse/fine point cloud registration algorithms. Based on structural coordinates, the crack segments from each scope were spliced and fused, and the complete crack morphology was plotted in the structural map. Field tests show that the identification network can overcome background interference and accurately distinguish the weld structure of specific regions by introducing the deep Hough transform to learn the scene semantics of steel box girders. By means of the obvious linear features of welds, the registration errors are controlled to about 1 mm with sparse point clouds. The fatigue crack perception performance is not sensitive to the camera poses. Under different lenses, the average localization error is within 5 mm, and the length measurement error is only about 1%.
-
Key words:
- fatigue crack /
- localization weld /
- point cloud registration /
- Hough transform
-
图 5 基于二分图匹配的定位焊缝识别性能评价
Figure 5. Evaluation of localization weld identification performance based on bipartite graph matching
图 7 相机空间内的裂纹与焊缝三维重建
Figure 7. Three-dimensional reconstruction of cracks and welds in camera space
图 8 焊缝辅助配准的疲劳裂纹局部定位
Figure 8. Local localization of fatigue crack with registration assisted by weld
图 10 不同相机姿态下的裂纹感知表现
Figure 10. Crack perception performance under different camera poses
表 1 测试集上检测结果
焊缝 $\rho_{\mathrm{Pre}} $ $\rho_{\mathrm{Rec}} $ $\rho_{\mathrm{F}} $ S $\rho_{\mathrm{mIoU}} $ $\tau^* $ T 0.970 0.970 0.970 0.958 0.739 0.340 H 0.936 0.946 0.939 0.894 0.739 0.020 V 0.902 0.934 0.913 0.923 0.739 0.240 
下载: 导出CSV
-
[1] 张清华, 李俊, 袁道云, 等. 深圳至中山跨江通道钢桥面板结构疲劳试验研究[J]. 土木工程学报, 2020, 53(11): 102-115.Zhang Qinghua, Li Jun, Yuan Daoyun, et al. Fatigue model tests of orthotropic steel bridge deck of Shenzhen-Zhongshan Link[J]. China Civil Engineering Journal, 2020, 53(11): 102-115. [2] 袁周致远, 吉伯海, 杜彦良, 等. 变幅荷载下钢桥疲劳裂纹扩展规律及长度预测方法研究[J]. 土木工程学报, 2023, 56(10): 43-51. doi: 10.15951/j.tmgcxb.22050485Yuan Zhouzhiyuan, Ji Bohai, Du Yanliang, et al. Investigation on fatigue crack propagation law and its length prediction method for steel bridge under variable amplitude load[J]. China Civil Engineering Journal, 2023, 56(10): 43-51. doi: 10.15951/j.tmgcxb.22050485 [3] 刘宇飞, 樊健生, 聂建国, 等. 结构表面裂缝数字图像法识别研究综述与前景展望[J]. 土木工程学报, 2021, 54(6): 79-98.Liu Yufei, Fan Jiansheng, Nie Jianguo, et al. Review and prospect of digital-image-based crack detection of structure surface[J]. China Civil Engineering Journal, 2021, 54(6): 79-98. [4] Ali R, Chuah J H, Abu Talip M S, et al. Crack segmentation network using additive attention gate: CSN-II[J]. Engineering Applications of Artificial Intelligence, 2022, 114: 105130. doi: 10.1016/j.engappai.2022.105130 [5] Xu Y, Fan Y L, Bao Y Q, et al. Task-aware meta-learning paradigm for universal structural damage segmentation using limited images[J]. Engineering Structures, 2023, 284: 115917. doi: 10.1016/j.engstruct.2023.115917 [6] 褚鸿鹄, 袁华青, 龙砺芝, 等. 基于Transformer的高分辨率桥梁裂缝图像级联分割方法[J]. 中国公路学报, 2024, 37(2): 65-76. doi: 10.19721/j.cnki.1001-7372.2024.02.006Chu Honghu, Yuan Huaqing, Long Lizhi, et al. A transformer-based cascade method for segmenting bridge cracks from high-resolution images[J]. China Journal of Highway and Transport, 2024, 37(2): 65-76. doi: 10.19721/j.cnki.1001-7372.2024.02.006 [7] Ye W L, Ren J J, Zhang A A, et al. Automatic pixel-level crack detection with multi-scale feature fusion for slab tracks[J]. Computer-Aided Civil and Infrastructure Engineering, 2023, 38(18): 2648-2665. doi: 10.1111/mice.12984 [8] Pan Z H, Guan J C, Yang X, et al. One-stage 3D profile-based pavement crack detection and quantification[J]. Automation in Construction, 2023, 153: 104946. doi: 10.1016/j.autcon.2023.104946 [9] Zhang H C, Zou Y, Del Rey Castillo E, et al. Detection of RC spalling damage and quantification of its key properties from 3D point cloud[J]. KSCE Journal of Civil Engineering, 2022, 26(5): 2023-2035. doi: 10.1007/s12205-022-0890-y [10] Yuan C, Xiong B, Li X Q, et al. A novel intelligent inspection robot with deep stereo vision for three-dimensional concrete damage detection and quantification[J]. Structural Health Monitoring, 2022, 21(3): 788-802. doi: 10.1177/14759217211010238 [11] Gao T, Yuanzhou Z, Ji B H, et al. Vision-based fatigue crack automatic perception and geometric updating of finite element model for welded joint in steel structures[J]. Computer-Aided Civil and Infrastructure Engineering, 2024, 39(11): 1659-1675. doi: 10.1111/mice.13166 [12] Kong Q Z, Gu J X, Xiong B, et al. Vision-aided three-dimensional damage quantification and finite element model geometric updating for reinforced concrete structures[J]. Computer-Aided Civil and Infrastructure Engineering, 2023, 38(17): 2378-2390. doi: 10.1111/mice.12967 [13] Zhang C G, Shu J P, Shao Y, et al. Automated generation of FE models of cracked RC beams based on 3D point clouds and 2D images[J]. Journal of Civil Structural Health Monitoring, 2022, 12(1): 29-46. doi: 10.1007/s13349-021-00525-5 [14] Hu K W, Chen Z, Kang H W, et al. 3D vision technologies for a self-developed structural external crack damage recognition robot[J]. Automation in Construction, 2024, 159: 105262. doi: 10.1016/j.autcon.2023.105262 [15] Xie E Z, Wang W H, Yu Z D, et al. SegFormer: simple and efficient design for semantic segmentation with transformers[EB/OL]. 2021-05-31. https://arxiv.org/abs/2105.15203 [16] Deng J, Dong W, Socher R, et al. ImageNet: a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2009: 248-255. [17] Lee J T, Kim H U, Lee C, et al. Semantic line detection and its applications[C]//2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2017: 3249-3257. [18] Kuhn H W. The Hungarian method for the assignment problem[J]. Naval Research Logistics Quarterly, 1955, 2(1/2): 83-97. [19] Cipolla R, Gal Y, Kendall A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake Piscataway: IEEE, 2018: 7482-7491. [20] Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327. doi: 10.1109/TPAMI.2018.2858826 [21] Kuehnle J U, Xue Z, Stotz M, et al. Grasping in Depth maps of time-of-flight cameras[C]//2008 International Workshop on Robotic and Sensors Environments. Piscataway: IEEE, 2008: 132-137. [22] Sun F, Choi Y K, Yu Y Z, et al. Medial meshes–a compact and accurate representation of medial axis transform[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 22(3): 1278-1290. doi: 10.1109/TVCG.2015.2448080 [23] Bereczky N, Duch A, Németh K, et al. Quad-kd trees: a general framework for kd trees and quad trees[J]. Theoretical Computer Science, 2016, 616: 126-140. doi: 10.1016/j.tcs.2015.12.030 [24] Xin M T, Li B, Wei X, et al. Rapid registration method by using partial 3D point clouds[J]. Optik, 2021, 246: 167764. doi: 10.1016/j.ijleo.2021.167764 [25] Besl P J, Mckay N D. A method for registration of 3-D shapes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(2): 239-256. doi: 10.1109/34.121791 -
下载:
点击查看大图
计量
- 文章访问数: 1
- HTML全文浏览量: 0
- PDF下载量: 0
- 被引次数: 0