• ISSN 0258-2724
  • CN 51-1277/U
  • EI Compendex
  • Scopus
  • Indexed by Core Journals of China, Chinese S&T Journal Citation Reports
  • Chinese S&T Journal Citation Reports
  • Chinese Science Citation Database
GUO Lie, GE Pingshu, WANG Xiao, WANG Dongxing. Visual Simultaneous Localization and Mapping Algorithm Based on Convolutional Neural Network to Optimize Loop Detection[J]. Journal of Southwest Jiaotong University, 2021, 56(4): 706-712, 768. doi: 10.3969/j.issn.0258-2724.20190723
Citation: GUO Lie, GE Pingshu, WANG Xiao, WANG Dongxing. Visual Simultaneous Localization and Mapping Algorithm Based on Convolutional Neural Network to Optimize Loop Detection[J]. Journal of Southwest Jiaotong University, 2021, 56(4): 706-712, 768. doi: 10.3969/j.issn.0258-2724.20190723

Visual Simultaneous Localization and Mapping Algorithm Based on Convolutional Neural Network to Optimize Loop Detection

doi: 10.3969/j.issn.0258-2724.20190723
  • Received Date: 19 Jul 2019
  • Rev Recd Date: 08 Mar 2020
  • Available Online: 01 Apr 2021
  • Publish Date: 15 Aug 2021
  • Traditional visual SLAM (simultaneous localization and mapping) without loop detection may lead to error accumulation. Even if there exits loop detection, it is unable to be applied to the lightweight applications because of its low accuracy and efficiency. Thus, a visual SLAM with loop detection optimization is studied. In the front-end estimation, ORB (oriented fast and rotated brief) feature points were abstracted and matched. PnP (perspective-n-point) was solved for the successful matched point to estimate the camera motion and screen out the key frame images. In the back-end optimization, SqueezeNet convolution neural network (CNN) was used to extract the feature vectors. The cosine similarities were calculated to determine whether there were loops or not. If there was a loop, the corresponding constraint was added to the posture graph. Then the global posture was optimized by using the graph optimization theory. Finally, tests and comparisons were conducted on the data sets produced by our research group and the public data sets of TUM. The results show that the proposed algorithm can detect loops successfully and add constraints to global trajectory optimization compared with the non-loop detection algorithm. Compared with the traditional word bag method, the recall rate of this method can be increased by 21% and the calculation time can be reduced by 74% under the same loop detection accuracy. Compared with RGB-D SLAM algorithm, the error of this method can be reduced by 29%.

     

  • 高翔, 张涛. 视觉SLAM十四讲[M]. 北京: 电子工业出版社, 2017: 9-32.
    蔡军,陈科宇,张毅. 基于Kinect的改进移动机器人视觉SLAM[J]. 智能系统学报,2018,13(5): 734-740.

    CAI Jun, CHEN Keyu, ZHANG Yi. Improved V-SLAM for mobile robots based on Kinect[J]. CAAI Transactions on Intelligent Systems, 2018, 13(5): 734-740.
    吕宪伟. 基于RGB-D数据的SLAM算法研究[D]. 北京: 北京理工大学, 2016.
    DAVISON A J, REID I D, MOLTON N D, et al. MonoSLAM:real-time single camera SLAM[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6): 1052-1067. doi: 10.1109/TPAMI.2007.1049
    KLEIN G, MURRAY D. Parallel tracking and mapping for small AR workspaces[C]//6th IEEE and ACM International Symposium on Mixed and Augmented Reality. Piscataway: IEEE, 2007: 225-234.
    MUR-ARTAL R, MONTIEL J M M, TARDOS J D. ORB-SLAM: a versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5): 1147-1163. doi: 10.1109/TRO.2015.2463671
    徐晓苏,代维,杨博,等. 室内环境下基于图优化的视觉惯性SLAM方法[J]. 中国惯性技术学报,2017,25(3): 313-319.

    XU Xiaosu, DAI Wei, YANG Bo, et al. Visual-aid inertial SLAM method based on graph optimization in indoor[J]. Journal of Chinese Inertial Technology, 2017, 25(3): 313-319.
    余杰. 基于ORB关键帧闭环检测算法的SLAM方法研究[D]. 杭州: 浙江工业大学, 2016.
    RUBLEE E, RABAUG V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C]// International Conference on Computer Vision. Piscataway: IEEE, 2011: 2564-2571.
    SCARAMUZZA D, FRAUNDORFER F. Visual odometry[J]. IEEE Robotics and Automation Magazine, 2011, 18(4): 80-92. doi: 10.1109/MRA.2011.943233
    LOWE D G. Object recognition from local scale-invariant features[C]//International Conference on Computer Vision. Piscataway: IEEE, 1999: 1150-1157
    席志红,李爽,甘兴利. PnP算法在室内定位中的应用[J]. 无线电工程,2017,47(10): 39-44.

    XI Zhihong, LI Shuang, GAN Xingli. PnP solution applied in indoor location[J]. Radio Engineering, 2017, 47(10): 39-44.
    刘国忠,胡钊政. 基于SURF和ORB全局特征的快速闭环检测[J]. 机器人,2017,39(1): 36-45.

    LIU Guozhong, HU Zhaozheng. Fast loop closure detection based on holistic features from SURF and ORB[J]. Robot, 2017, 39(1): 36-45.
    GAO X, ZHANG T. Loop closure detection for visual slam systems using deep neural networks[C]//Chinese Control Conference. Piscataway: IEEE, 2015: 5851-5856.
    SHANG W, SOHN K, ALMEIDA D, et al. Understanding and improving convolutional neural networks via concatenated rectified linear units[C]//International Conference on Machine Learning. Princeton: IMLS, 2016: 3276-3284.
    KUMMERLE R, GRISETTI G, STRASDAT H, et al. G2o: a general framework for graph optimization[C]// IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2011: 3607-3613.
    JIA Y Q, SHELHAMER E, JEFF D. Caffe: convolutional architecture for fast feature embedding[C]//ACM International Conference on Multimedia. New York: ACM, 2014: 675-678.
    STURM J, ENGELHARD N, ENDRES F, et al. A benchmark for the evaluation of RGB-D SLAM systems[C]//International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2012: 573-580.
    ENDRES F, HESS J, STURM J, et al. 3-D mapping with an RGB-D camera[J]. IEEE Transactions on Robotics, 2014, 30(1): 177-187. doi: 10.1109/TRO.2013.2279412
  • Relative Articles

    [1]WANG Huiqin, GUO Ruili, HE Yongqiang, LIU Bincan. NGO-Based CNN-BiLSTM-AM Model for Landslide Displacement Prediction[J]. Journal of Southwest Jiaotong University. doi: 10.3969/j.issn.0258-2724.20240550
    [2]YANG Yanchun, YAN Yan, WANG Ke. Infrared and Visible Image Fusion Based on Attention Mechanism and Illumination-Aware Network[J]. Journal of Southwest Jiaotong University, 2024, 59(5): 1204-1214. doi: 10.3969/j.issn.0258-2724.20230529
    [3]ZHANG Chunxiang, TANG Libo, GAO Xueyao. Word Sense Disambiguation Based on Semi-Supervised Convolutional Neural Networks[J]. Journal of Southwest Jiaotong University, 2022, 57(1): 11-17, 27. doi: 10.3969/j.issn.0258-2724.20200105
    [4]XING Yulong, WANG Jian, ZHAO Huibing, ZHU Linfu. Cab Signal Denoising Process Based on Fully Convolutional Networks[J]. Journal of Southwest Jiaotong University, 2021, 56(2): 444-450. doi: 10.3969/j.issn.0258-2724.20191111
    [5]CUI Hongtao, CAO Ke, ZHANG Hu, CUI Xiao. Weather Classification in Traffic Scene Based on Joint Voting Network[J]. Journal of Southwest Jiaotong University, 2021, 56(3): 579-586. doi: 10.3969/j.issn.0258-2724.20200084
    [6]XU Chang’an, NI Shaoquan, CHEN Dingjun. Collaborative Optimization for Timetable and Maintenance Window Based on Two-Stage Algorithm[J]. Journal of Southwest Jiaotong University, 2020, 55(4): 882-888. doi: 10.3969/j.issn.0258-2724.20180577
    [7]YUAN Fei, ZHAO Xuyan, WANG Yige, ZHAO Zhisheng. Smoke Recognition Algorithm Based on Lightweight Convolutional Neural Network[J]. Journal of Southwest Jiaotong University, 2020, 55(5): 1111-1116, 1132. doi: 10.3969/j.issn.0258-2724.20190777
    [8]ZHOU Wenliang, LI Peng, TIAN Junli, DENG Lianbo. Optimization of Train Timetable for Intercity Railway Based on Coordinated Operation of Multi-periodic Trains[J]. Journal of Southwest Jiaotong University, 2019, 54(4): 831-839. doi: 10.3969/j.issn.0258-2724.20170153
    [9]XIANG Yu, CONG Deming, ZHANG Yang, YUAN Fei. Two-Stream Neural Network Fusion Model for Highway Fog Detection[J]. Journal of Southwest Jiaotong University, 2019, 54(1): 173-179. doi: 10.3969/j.issn.0258-2724.20180205
    [10]HOU Jin, LÜ Zhiliang, XU Mao, WU Peijun, LIU Yuling, ZHANG Xiaoyu, CHENG Zeng. Combined Neural Networks Based on Deep Learning for Signal Detection in Aeronautical Communications[J]. Journal of Southwest Jiaotong University, 2019, 54(4): 863-869, 878. doi: 10.3969/j.issn.0258-2724.20180164
    [11]XIE Yanmin, TANG Wei, HUANG Renyong, XIONG Wencheng, ZHUO Dezhi. Drawbead Optimisation in Stamping Using SA-RBF Neural Networks[J]. Journal of Southwest Jiaotong University, 2017, 30(5): 970-976,993. doi: 10.3969/j.issn.0258-2724.2017.05.018
    [12]ZHANG Xiaobing, NI Shaoquan, PAN Jinshan. Optimization of Train Diagram Structure for High-Speed Railway[J]. Journal of Southwest Jiaotong University, 2016, 29(5): 938-943. doi: 10.3969/j.issn.0258-2724.2016.05.017
    [13]XIE Yanmin, HE Yujun, TIAN Yin. Optimization of Variable Blank Holder Forces in Sheet Metal Forming Based on RBF Neural Network Model[J]. Journal of Southwest Jiaotong University, 2016, 29(1): 121-127. doi: 10.3969/j.issn.0258-2724.2016.01.018
    [14]GUANQin-chuan. Fuzzy Multi-objective Optimization Based on Neural Networks[J]. Journal of Southwest Jiaotong University, 2002, 15(3): 338-342.
    [15]ZHAO Zheng-jia, HUANGHong-zhong, CHENXin. A Genetic-Neural Network Algorithm in Optimum Design[J]. Journal of Southwest Jiaotong University, 2000, 13(1): 65-68.
  • Cited by

    Periodical cited type(9)

    1. 兰珂,曹计划. 三维虚拟视频关键帧特征提取算法研究. 计算机仿真. 2025(01): 188-191+461 .
    2. 曾琪,杨真. 考虑多尺度纹理特征的红外传感图像频域增强. 传感技术学报. 2024(04): 652-657 .
    3. 刘焕钊,蒋林,郭宇飞,张文俊,吴艳霞,张琼玉. 基于三维点云转换视觉图像的回环检测算法. 组合机床与自动化加工技术. 2023(04): 91-95+99 .
    4. 苏子旸,张策,张茹,张展,张婧,吕为工. 视觉同步定位与建图系统中回环检测研究进展. 计算机技术与发展. 2023(04): 1-8 .
    5. 柳宁,崔小军,绳鹏鹏,张超,雷兴. 基于SLAM的电厂巡检机器人导航技术实现. 自动化与仪器仪表. 2023(07): 214-218 .
    6. 张耀,吴一全,陈慧娴. 基于深度学习的视觉同时定位与建图研究进展. 仪器仪表学报. 2023(07): 214-241 .
    7. 孟繁,周馨曌,吴烽云,邹天龙. 基于视觉SLAM的水果采摘机器人研究进展. 自动化与信息工程. 2023(05): 8-13+31 .
    8. 曾庆化,罗怡雪,孙克诚,李一能,刘建业. 视觉及其融合惯性的SLAM技术发展综述. 南京航空航天大学学报. 2022(06): 1007-1020 .
    9. 梅立雪. 基于深度卷积的神经网络的SLAM算法研究. 造纸装备及材料. 2022(10): 70-72 .

    Other cited types(18)

  • Created with Highcharts 5.0.7Amount of accessChart context menuAbstract Views, HTML Views, PDF Downloads StatisticsAbstract ViewsHTML ViewsPDF Downloads2024-082024-092024-102024-112024-122025-012025-022025-032025-042025-052025-062025-07010203040
    Created with Highcharts 5.0.7Chart context menuAccess Class DistributionFULLTEXT: 28.1 %FULLTEXT: 28.1 %META: 65.7 %META: 65.7 %PDF: 6.3 %PDF: 6.3 %FULLTEXTMETAPDF
    Created with Highcharts 5.0.7Chart context menuAccess Area Distribution其他: 9.9 %其他: 9.9 %其他: 0.5 %其他: 0.5 %Central District: 0.8 %Central District: 0.8 %China: 0.6 %China: 0.6 %上海: 1.9 %上海: 1.9 %东京: 0.1 %东京: 0.1 %东莞: 1.3 %东莞: 1.3 %临汾: 0.2 %临汾: 0.2 %临沂: 0.1 %临沂: 0.1 %丽水: 0.2 %丽水: 0.2 %佛山: 0.1 %佛山: 0.1 %北京: 5.2 %北京: 5.2 %南京: 1.5 %南京: 1.5 %南宁: 0.1 %南宁: 0.1 %南昌: 0.3 %南昌: 0.3 %合肥: 0.8 %合肥: 0.8 %吉林: 0.1 %吉林: 0.1 %哥伦布: 0.1 %哥伦布: 0.1 %嘉兴: 0.2 %嘉兴: 0.2 %天津: 1.3 %天津: 1.3 %太原: 0.1 %太原: 0.1 %官坑: 0.1 %官坑: 0.1 %宣城: 0.5 %宣城: 0.5 %山东省: 0.2 %山东省: 0.2 %广州: 0.3 %广州: 0.3 %延安: 0.2 %延安: 0.2 %张家口: 2.2 %张家口: 2.2 %德阳: 0.5 %德阳: 0.5 %成都: 5.5 %成都: 5.5 %扬州: 0.1 %扬州: 0.1 %景德镇: 0.1 %景德镇: 0.1 %杭州: 1.4 %杭州: 1.4 %桂林: 0.1 %桂林: 0.1 %武汉: 1.5 %武汉: 1.5 %池州: 0.2 %池州: 0.2 %沈阳: 1.8 %沈阳: 1.8 %沧州: 0.2 %沧州: 0.2 %洛杉矶: 0.1 %洛杉矶: 0.1 %洛阳: 0.3 %洛阳: 0.3 %济南: 0.1 %济南: 0.1 %海口: 0.2 %海口: 0.2 %淮南: 0.1 %淮南: 0.1 %深圳: 0.8 %深圳: 0.8 %温州: 0.2 %温州: 0.2 %漯河: 0.3 %漯河: 0.3 %漳州: 0.2 %漳州: 0.2 %焦作: 0.2 %焦作: 0.2 %珠海: 0.3 %珠海: 0.3 %石家庄: 3.5 %石家庄: 3.5 %秦皇岛: 0.5 %秦皇岛: 0.5 %芒廷维尤: 15.7 %芒廷维尤: 15.7 %芝加哥: 0.9 %芝加哥: 0.9 %苏州: 0.3 %苏州: 0.3 %衡水: 0.5 %衡水: 0.5 %西宁: 23.9 %西宁: 23.9 %西安: 0.7 %西安: 0.7 %贵阳: 0.3 %贵阳: 0.3 %运城: 0.8 %运城: 0.8 %重庆: 0.9 %重庆: 0.9 %长春: 0.1 %长春: 0.1 %长沙: 5.1 %长沙: 5.1 %雅安: 0.1 %雅安: 0.1 %青岛: 2.5 %青岛: 2.5 %鞍山: 0.2 %鞍山: 0.2 %马湾: 0.2 %马湾: 0.2 %其他其他Central DistrictChina上海东京东莞临汾临沂丽水佛山北京南京南宁南昌合肥吉林哥伦布嘉兴天津太原官坑宣城山东省广州延安张家口德阳成都扬州景德镇杭州桂林武汉池州沈阳沧州洛杉矶洛阳济南海口淮南深圳温州漯河漳州焦作珠海石家庄秦皇岛芒廷维尤芝加哥苏州衡水西宁西安贵阳运城重庆长春长沙雅安青岛鞍山马湾

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(9)  / Tables(4)

    Article views(828) PDF downloads(56) Cited by(27)
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return