Cloud Removal Method for Multi-Temporal Remote Sensing Image Based on Factor Group Sparsity Regularization
-
摘要:
多云天气会导致多时相遥感图像（multi-temporal remote sensing image，MTRSI）存在云覆盖与信息缺失问题，进而影响后续应用效能. 近年来，基于低秩矩阵/张量分解和稀疏正则化的去云方法，忽略了图像不同波段间与不同时间维度间的一致性特征. 为此，本文在低秩矩阵分解框架下，提出了基于因子组稀疏正则的新型MTRSI去云模型. 具体地，该模型利用低秩矩阵分解刻画干净图像的全局时空相关性；通过对多维度数据对应的因子施加组稀疏正则，精准捕捉与约束跨波段、跨时间的空间平滑区域一致性. 进一步地，设计一种内嵌交替方向乘子法的近端交替最小化算法，将原始优化问题分解为低秩约束与稀疏正则项优化两个子问题，通过交替迭代更新，实现对原始问题的高效求解. 在仿真实验中，相较于次优方法，所提去云方法的平均峰值信噪比提升了18.01%，平均光谱角制图减少了43.01%，平均结构相似度增加了4.1×10−3，平均相关系数增加了2.3×10−3，所需运行时间减少了78.43%；同时，在真实场景实验中也取得了更好的去云效果.
Abstract:Cloudy weather causes cloud coverage and information loss in multi-temporal remote sensing image, affecting the subsequent application performance. In recent years, cloud removal methods based on low-rank matrix/tensor decomposition and sparse regularization have ignored the consistency features across different bands and different temporal dimensions of image. To this end, under the framework of low-rank matrix decomposition, a novel cloud removal model for multi-temporal remote sensing image based on factor group sparsity regularization was proposed. Specifically, global spatiotemporal correlations of clean image were characterized by the model using low-rank matrix decomposition; by imposing group sparsity regularization on the factors corresponding to multi-dimensional data, the consistency of spatially smooth regions across bands and time was accurately captured and constrained. Furthermore, a proximal alternating minimization algorithm embedded with the alternating direction method of multipliers was designed to decompose the original optimization problem into two sub-problems, namely low-rank constraint and sparse regularization term optimization, which were updated alternately and iteratively, thereby achieving an efficient solution to the original problem. In simulation experiments, compared with the suboptimal method, the average peak signal-to-noise ratio of the proposed cloud removal method increases by 18.01%; the average spectral angle mapper decreases by 43.01%; the average structural similarity increases by 4.1 × 10−3; the average correlation coefficient increases by 2.3 × 10−3, and the required running time decreases by 78.43%. Meanwhile, better cloud removal results are also achieved in real-scene experiments.
-
图 1 基于因子组稀疏正则的遥感图像去云方法流程
Figure 1. Flowchart of remote sensing image cloud removal method based on factor group sparsity regularization
图 2 MTRSI差分图像的GS特性示意
Figure 2. Schematic diagram of GS characteristics of differential image of MTRSI
图 3 不同方法在巴西数据集上的去云结果
Figure 3. Cloud removal results of different methods on Brazil dataset
图 4 不同方法在塔公数据集上的去云结果
Figure 4. Cloud removal results of different methods on Tagong dataset
表 1 不同方法在模拟数据集上的定量比较结果
Table 1. Quantitative comparison results of different methods on simulated datasets
数据集 序列 指标 观测结果 Regression TNN TVLRSDC TVTR MT 本文方法 摩洛哥
Morocco1 PSNR 12.394 1 46.174 7 48.268 4 46.351 0 44.928 9 46.316 5 53.929 3 SSIM 0.891 7 0.998 0 0.995 9 0.997 3 0.987 9 0.988 5 0.998 5 SAM 2.571 1 0.310 9 0.129 4 0.113 7 0.140 4 0.099 7 0.073 2 CC 0.492 6 0.999 7 0.999 4 0.999 1 0.998 6 0.995 1 0.999 8 2 PSNR 11.432 3 43.913 5 44.457 9 43.711 7 40.841 8 41.507 8 52.229 9 SSIM 0.826 5 0.992 0 0.987 7 0.992 1 0.971 0 0.980 2 0.997 6 SAM 8.425 2 0.331 4 0.189 2 0.199 6 0.319 6 0.252 6 0.125 7 CC 0.391 9 0.999 0 0.998 7 0.998 5 0.997 2 0.989 3 0.999 8 3 PSNR 6.428 2 38.383 1 41.565 8 42.815 4 37.217 9 28.020 5 45.866 9 SSIM 0.672 7 0.981 2 0.977 3 0.987 7 0.943 8 0.903 3 0.988 1 SAM 9.689 5 0.387 9 0.495 4 0.415 8 0.516 3 0.670 4 0.265 2 CC 0.209 1 0.994 0 0.997 1 0.997 8 0.989 8 0.705 5 0.998 7 运行时间/s — — 40.612 0 156.611 4 161.150 5 337.869 4 23.187 8 巴西
Brazil1 PSNR 10.754 2 39.911 1 36.322 7 34.758 2 41.339 3 38.849 7 49.343 9 SSIM 0.812 1 0.991 9 0.985 0 0.987 1 0.975 1 0.973 0 0.996 6 SAM 9.073 2 0.738 0 0.444 4 0.323 4 0.410 5 0.428 8 0.166 6 CC 0.423 0 0.997 8 0.993 2 0.992 4 0.998 2 0.991 5 0.999 7 2 PSNR 11.577 6 39.017 7 46.159 8 47.755 3 43.951 5 45.322 2 49.584 1 SSIM 0.842 2 0.992 7 0.992 7 0.996 1 0.981 6 0.989 0 0.996 6 SAM 8.861 5 1.015 9 0.282 2 0.216 9 0.328 6 0.301 6 0.120 1 CC 0.434 5 0.997 7 0.999 4 0.999 4 0.999 0 0.998 3 0.999 7 3 PSNR 10.836 9 37.421 0 32.898 6 38.403 8 42.162 8 40.145 8 48.549 0 SSIM 0.839 0 0.989 6 0.975 5 0.987 2 0.975 5 0.980 5 0.994 6 SAM 7.701 5 0.839 4 0.980 0 0.324 7 0.354 6 0.356 8 0.157 3 CC 0.535 4 0.997 9 0.987 1 0.996 6 0.998 6 0.994 6 0.999 7 运行时间/s — — 40.528 0 68.220 4 273.197 0 344.564 7 25.289 8 
下载: 导出CSV
表 2 本文方法对不同比例云污染的去除结果
Table 2. Removal results of cloud pollution with different proportions by proposed method
覆盖比例/
%PSNR SSIM SAM CC 12.5 50.671 6 0.996 6 0.114 4 0.999 3 25.0 45.893 2 0.991 9 0.232 1 0.998 0 50.0 39.201 5 0.978 5 0.463 3 0.990 7
下载: 导出CSV
表 3 本文方法对不同时相云污染的去除结果
Table 3. Removal results of cloud pollution in different temporal phases by proposed method
污染时相 PSNR SSIM SAM CC 1 50.628 3 0.994 7 0.14853 0.997 7 2 50.879 9 0.994 7 0.14823 0.997 6 3 50.767 6 0.994 8 0.14866 0.997 7
下载: 导出CSV
-
[1] Liu W C, Ma L, Chen H. Arbitrary-oriented ship detection framework in optical remote-sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(6): 937-941. doi: 10.1109/LGRS.2018.2813094 [2] Qin C, Wang X Q, Li G, et al. An improved attention-guided network for arbitrary-oriented ship detection in optical remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 6514805. [3] Zhang Z N, Zhang L, Wang Y, et al. ShipRSImageNet: a large-scale fine-grained dataset for ship detection in high-resolution optical remote sensing images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 8458-8472. doi: 10.1109/JSTARS.2021.3104230 [4] 阮永俭. 中高分辨率遥感植被物候信息提取与特征分析研究[J]. 测绘学报, 2023, 52(10): 1801.Ruan Yongjian. Extraction and feature analysis of vegetation phenology information from medium-high resolution remote sensing data[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(10): 1801. [5] Shi S N, Zhong Y F, Zhao J, et al. Land-use/land-cover change detection based on class-prior object-oriented conditional random field framework for high spatial resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5600116. doi: 10.1109/tgrs.2020.3034373 [6] Yuan G, Wang Y J, Zhao F, et al. Spatiotemporal correlation characteristics between thermal infrared remote sensing obtained surface thermal anomalies and reconstructed 4-D temperature fields of underground coal fires[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 4506318. [7] 李玉美, 郭庆华, 万波, 等. 基于激光雷达的自然资源三维动态监测现状与展望[J]. 遥感学报, 2021, 25(1): 381-402.Li Yumei, Guo Qinghua, Wan Bo, et al. Current status and prospect of three-dimensional dynamic monitoring of natural resources based on LiDAR[J]. National Remote Sensing Bulletin, 2021, 25(1): 381-402. [8] 李欢, 万玮, 冀锐, 等. 中国卫星遥感地表水资源监测能力分析与展望[J]. 遥感学报, 2023, 27(7): 1554-1573.Li Huan, Wan Wei, Ji Rui, et al. Inspects and prospects of satellite remote sensing monitoring ability for land surface water in China[J]. National Remote Sensing Bulletin, 2023, 27(7): 1554-1573. [9] Du H Q, Mao F J, Li X J, et al. Mapping global bamboo forest distribution using multisource remote sensing data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(5): 1458-1471. doi: 10.1109/JSTARS.2018.2800127 [10] 唐尧, 王立娟, 马国超, 等. 利用国产遥感卫星进行金沙江高位滑坡灾害灾情应急监测[J]. 遥感学报, 2019, 23(2): 252-261.Tang Yao, Wang Lijuan, Ma Guochao, et al. Emergency monitoring of high-level landslide disasters in Jinsha River using domestic remote sensing satellites[J]. Journal of Remote Sensing, 2019, 23(2): 252-261. [11] 秦军, 曹云刚, 耿娟. 汶川地震灾区道路损毁度遥感评估模型[J]. 西南交通大学学报, 2010, 45(5): 768-774.Qin Jun, Cao Yungang, Geng Juan. Evaluation model for damage extent of roads in Wenchuan earthquake-stricken areas based on remote sensing information[J]. Journal of Southwest Jiaotong University, 2010, 45(5): 768-774. [12] 陈晓勇, 刘波, Aung Kyaw Tun. 遥感在缅甸飓风灾害土地覆盖分类中的应用[J]. 西南交通大学学报, 2011, 46(4): 701-705.Chen Xiaoyong, Liu Bo, Tun Aung Kyaw. Application of remote sensing to land cover change induced by nargis hurricane in Myanmar[J]. Journal of Southwest Jiaotong University, 2011, 46(4): 701-705. [13] Ji T Y, Chu D L, Zhao X L, et al. A unified framework of cloud detection and removal based on low-rank and group sparse regularizations for multitemporal multispectral images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5303015. doi: 10.1109/tgrs.2022.3152630 [14] Chen Y, He W, Yokoya N, et al. Blind cloud and cloud shadow removal of multitemporal images based on total variation regularized low-rank sparsity decomposition[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 157: 93-107. doi: 10.1016/j.isprsjprs.2019.09.003 [15] 郑凯, 李建胜, 王俊强, 等. DCLS-GAN: 利用生成对抗网络的天绘一号卫星高原地区影像去云方法[J]. 测绘学报, 2021, 50(2): 248-259.Zheng Kai, Li Jiansheng, Wang Junqiang, et al. DCLS-GAN: Cloud removal method for plateau area of TH-1 satellite image[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2): 248-259. [16] Tseng D C, Tseng H T, Chien C L. Automatic cloud removal from multi-temporal SPOT images[J]. Applied Mathematics and Computation, 2008, 205(2): 584-600. doi: 10.1016/j.amc.2008.05.050 [17] Li X H, Wang L Y, Cheng Q, et al. Cloud removal in remote sensing images using nonnegative matrix factorization and error correction[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 148: 103-113. doi: 10.1016/j.isprsjprs.2018.12.013 [18] Xu M, Jia X P, Pickering M, et al. Cloud removal based on sparse representation via multitemporal dictionary learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(5): 2998-3006. doi: 10.1109/TGRS.2015.2509860 [19] Zhang Q, Yuan Q Q, Li J, et al. Thick cloud and cloud shadow removal in multitemporal imagery using progressively spatio-temporal patch group deep learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 162: 148-160. doi: 10.1016/j.isprsjprs.2020.02.008 [20] Zhang Q, Yuan Q Q, Li Z W, et al. Combined deep prior with low-rank tensor SVD for thick cloud removal in multitemporal images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 177: 161-173. doi: 10.1016/j.isprsjprs.2021.04.021 [21] Czerkawski M, Upadhyay P, Davison C, et al. Deep internal learning for inpainting of cloud-affected regions in satellite imagery[J]. Remote Sensing, 2022, 14(6): 1342. doi: 10.3390/rs14061342 [22] Long C J, Li X H, Jing Y H, et al. Bishift networks for thick cloud removal with multitemporal remote sensing images[J]. International Journal of Intelligent Systems, 2023, 2023: 9953198. doi: 10.1155/2023/9953198 [23] Kiers H A L. Towards a standardized notation and terminology in multiway analysis[J]. Journal of Chemometrics, 2000, 14(3): 105-122. doi: 10.1002/1099-128X(200005/06)14:3<105::AID-CEM582>3.0.CO;2-I [24] Tucker L R. Some mathematical notes on three-mode factor analysis[J]. Psychometrika, 1966, 31(3): 279-311. doi: 10.1007/BF02289464 [25] Kilmer M E, Martin C D. Factorization strategies for third-order tensors[J]. Linear Algebra and Its Applications, 2011, 435(3): 641-658. doi: 10.1016/j.laa.2010.09.020 [26] Oseledets I V. Tensor-train decomposition[J]. SIAM Journal on Scientific Computing, 2011, 33(5): 2295-2317. doi: 10.1137/090752286 [27] Zhao Q B, Zhou G X, Xie S L, et al. Tensor ring decomposition[PP/OL]. V1. arXiv (2016-06-17)[2024-12-10]. https://doi.org/10.48550/arXiv.1606.05535. [28] Zheng Y B, Huang T Z, Zhao X L, et al. Fully-connected tensor network decomposition and its application to higher-order tensor completion[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(12): 11071-11078. doi: 10.1609/aaai.v35i12.17321 [29] Yokota T, Zhao Q B, Cichocki A. Smooth PARAFAC decomposition for tensor completion[J]. IEEE Transactions on Signal Processing, 2016, 64(20): 5423-5436. doi: 10.1109/TSP.2016.2586759 [30] Ji T Y, Yokoya N, Zhu X X, et al. Nonlocal tensor completion for multitemporal remotely sensed images’ inpainting[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(6): 3047-3061. doi: 10.1109/TGRS.2018.2790262 [31] Liu J, Musialski P, Wonka P, et al. Tensor completion for estimating missing values in visual data[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 208-220. doi: 10.1109/TPAMI.2012.39 [32] Zhang Z M, Aeron S. Exact tensor completion using t-SVD[J]. IEEE Transactions on Signal Processing, 2017, 65(6): 1511-1526. doi: 10.1109/TSP.2016.2639466 [33] Bengua J A, Phien H N, Tuan H D, et al. Efficient tensor completion for color image and video recovery: low-rank tensor train[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2466-2479. doi: 10.1109/TIP.2017.2672439 [34] He W, Yokoya N, Yuan L H, et al. Remote sensing image reconstruction using tensor ring completion and total variation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11): 8998-9009. doi: 10.1109/TGRS.2019.2924017 [35] Li L Y, Huang T Z, Zheng Y B, et al. Thick cloud removal for multitemporal remote sensing images: when tensor ring decomposition meets gradient domain fidelity[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5512414. doi: 10.1109/tgrs.2023.3276414 [36] Zheng W J, Zhao X L, Zheng Y B, et al. Spatial-spectral-temporal connective tensor network decomposition for thick cloud removal[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 199: 182-194. doi: 10.1016/j.isprsjprs.2023.04.006 [37] Lin J, Huang T Z, Zhao X L, et al. Robust thick cloud removal for multitemporal remote sensing images using coupled tensor factorization[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5406916. doi: 10.1109/tgrs.2022.3140800 [38] Wang J L, Zhao X L, Li H C, et al. Unsupervised domain factorization network for thick cloud removal of multitemporal remotely sensed images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5405912. -
下载:
点击查看大图
计量
- 文章访问数: 24
- HTML全文浏览量: 18
- PDF下载量: 0
- 被引次数: 0