Video Error Concealment Algorithm Based on Visual Saliency
-
摘要: 为了避免在差错掩盖过程中由于不当的替代块选择而引入新的可察觉失真，将视觉显著性与差错掩盖算法相结合，提出了一种基于视觉显著性的视频差错掩盖算法. 首先使用显著图检测算法将原始视频的每帧图像分为感兴趣区域和非感兴趣区域，接下来在解码端利用视频序列的显著性分布信息进行差错掩盖，保证候选替代块的选择范围被限制在与原始单元具有相似视觉显著性分布水平的区域内；其次在非感兴趣区域受损单元的掩盖算法中引入显著性降低操作算子，通过算子循环迭代来降低其替代块的强度和颜色对比度，使得掩盖后的图像区域既能保持较好的匹配性又具有低显著性；最后利用快速非局部均值去噪算法，对完成差错掩盖操作的视频序列进行去噪处理，以提高视频序列的整体视觉质量. 实验结果表明:在3%、5%、10%丢包率下，相对于传统的边界匹配算法，该算法掩盖后的序列的客观质量具有0.79～1.66 dB的提升；相对于解码端运动估计算法，该算法掩盖后的序列的客观质量具有0.39～1.55 dB的提升；在掩盖完成后使用非局部均值去噪算法，视频序列的客观质量有0.05～0.51 dB的提升.Abstract: In order to avoid new noticeable distortions owing to replacement information selected improperly during error concealment, this paper combines visual saliency with video processing and proposes a novel video error concealment algorithm based on visual saliency. Firstly, each video frame is divided into two parts: a region of interest (ROI) and non-ROI, by a suitable saliency detection method. The saliency information serves as an important reference to guide error concealment in the decoder, and limits the range of the replacement block to the region with similar saliency as the original image. Secondly, a saliency reduction operator is designed and applied iteratively in a loop to decrease the intensity and colour contrast of the replacement block of the non-ROI; then, the candidate replacement blocks will have good match and low saliency value. Finally, a low-complexity non-local means denoising method is used to reduce the imperfections of error concealment. The experimental results indicate that when the packet loss rate is 3%, 5%, and 10%, the proposed algorithm has increased 0.79–1.66 dB relative to the boundary matching algorithm, and 0.39–1.55 dB relative to decoder motion vector estimation. In addition, the performance of the algorithm combined with non-local means denoising improved by 0.05–0.51 dB compared to using the proposed algorithm alone.
-
图 2 LC、HC以及FT显著性检测算法的主观效果
Figure 2. Subjective results of 3 different salient region detection algorithms (LC，HC，and FT)
图 4 基于视觉显著性的差错掩盖算法流程
Figure 4. Flow chart of error concealment algorithm based on visual saliency
图 5 丢包率为10%，Football序列第82帧差错掩盖后的主观视觉质量
Figure 5. Subjective results for error concealment of the 82th frame of sequence Football when PLR is 10%
表 1 4种算法差错掩盖性能比较
Table 1. Comparison of 4 algorithms performance
丢包率/% 掩盖策略 测试序列 City Crew Foreman Football 3 BMA 34.17 34.75 35.18 31.94 DMVE 34.57 35.24 35.77 32.39 Pro 34.96 35.64 36.21 32.99 Pro+去噪 35.01 35.82 36.26 33.22 5 BMA 33.42 33.06 33.95 29.71 DMVE 33.87 33.61 34.82 30.06 Pro 34.19 34.09 35.33 30.99 Pro+去噪 34.29 34.43 35.43 31.36 10 BMA 32.24 31.86 32.36 27.34 DMVE 32.80 31.89 32.68 27.45 Pro 33.56 32.87 33.29 29.00 Pro+去噪 33.61 33.21 33.24 29.51 
下载: 导出CSV
-
LAM W M, REIBMAN A R, LIU B. Recovery of lost or erroneously received motion vectors[C]// IEEE International Conference on Acoustics, Speech, and Signal Processing. Minneapolis: IEEE, 1993: 417-420 ZHANG Jian, ARNOLD J F, FRATER M R. A cell-loss concealment technique for MPEG-2 coded video[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2000, 10(4): 659-665 XIONG Bing, Fan Xiaojiu, Zhu Ce, et al. Face region based conversational video coding[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2011, 21(7): 917-931 XU Mai, DENG Xin, LI Shengxi, et al. Region-of-interest based conversational HEVC coding with hierarchical perception model of face[J]. IEEE Journal of Selected Topics in Signal Processing, 2014, 8(3): 475-489 MAUNG H M, ARAMVITH S, MIYANAGA Y. Region-of-interest based error resilient method for HEVC video transmission[C]// International Symposium on Communications and Information Technologies. Nara: IEEE, 2016:241-244 HADIZADEH H, BAJIC I V. Saliency-aware video compression[J]. IEEE Transactions on Image Processing, 2014, 23(1): 19-33 ZHANG Zhewei, JING Tao, HAN Jingning, et al. A new rate control scheme for video coding based on region of interest[J]. IEEE Access, 2017, 5: 13677-13688 SHI Yunhui, YUE Shaoyuan, YIN Baocai, et al. A novel ROI-based rate control scheme for H.264[C]// Young Computer Scientists. [S.l.]: IEEE, 2008: 77-81 CHIANG J C, HSIEH C S, CHANG G, et al. Region-of-interest based rate control scheme with flexible quality on demand[J]. IEEE, 2010, 26(2): 238-242 HADIZADEH H, BAJIC I V. Saliency-preserving video compression[C]// IEEE International Conference on Multimedia and Expo. Barcelona: IEEE, 2011: 1-6 MEDDEB M, CAGNAZZO M, PESQUET-POPESCU B. Region-of-interest based rate control scheme for high efficiency video coding[C]// Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on. [S.l.]: IEEE, 2014: 7338-7342 WANG Bing, PENG Qiang, CHEN Jian, et al. A low-complexity error concealment algorithm for video transmission based on non-local means denoising[C]// Visual Communications and Image Processing. Chengdu: IEEE, 2016: 1-4 吴金建. 基于人类视觉系统的图像信息感知和图像质量评价[D]. 西安: 西安电子科技大学, 2014 ITTI L, KOCH C, NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on pattern analysis and machine intelligence, 1998, 20(11): 1254-1259 ZHAI Y, SHAH M. Visual attention detection in video sequences using spatiotemporal cues[C]//Proceedings of the 14th ACM international conference on Multimedia. Santa Barbara: ACM, 2006: 815-824 CHENG M M, MITRA N J, HUANG X, et al. Global contrast based salient region detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 569-582 ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency-tuned salient region detection[C]// IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 1597-1604 SEILER J, SCHOBERI M, KAUP A. Spatio-temporal error concealment in video by denoised temporal extrapolation refinement[C]// IEEE International Conference on Image Processing. Melbourne: IEEE, 2013: 1613-1616 HAGIWARA A, SUGIMOTO A, KAWAMOTO K. Saliency-based image editing for guiding visual attention[C]//Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interaction. Beijing: ACM, 2011: 43-48 Joint Video Team. JVT reference software JM19.0[CP/OL]. [2016-03-09]. http://iphome.hhi.de/suehring/tml/download -
下载:
点击查看大图
图(5) / 表(1)
计量
- 文章访问数: 451
- HTML全文浏览量: 236
- PDF下载量: 110
- 被引次数: 0