面向舰船目标检测的SAR图像数据PCGAN生成方法

潘磊; 郭宇诗; 李恒超; 王伟业; 李泽琛; 马天宇

doi:10.3969/j.issn.0258-2724.20210630

面向舰船目标检测的SAR图像数据PCGAN生成方法

doi: 10.3969/j.issn.0258-2724.20210630

潘磊^1,,
郭宇诗^2, ,,
李恒超^{3, 4},
王伟业³,
李泽琛³,
马天宇³

1.
中国电子科技集团公司第十研究所，四川成都 610036
2.
西南交通大学计算机与人工智能学院，四川成都 611756
3.
西南交通大学信息科学与技术学院，四川成都 611756
4.
西南交通大学综合交通大数据应用技术国家工程实验室，四川成都 611756

基金项目: 国家自然科学基金（62001437，61871335）；中央高校基本业务费专项资金（2682020ZT35）

详细信息

作者简介:
潘磊（1986—），男，工程师，博士，研究方向为图像处理与模式识别，E-mail：mapan.lei@163.com

通讯作者:
郭宇诗（1998—），女，工程师，研究方向为图像处理与模式识别，E-mail：ysguo1634@163.com

中图分类号: TP391.41；TP183
计量
- 文章访问数: 222
- HTML全文浏览量: 77
- PDF下载量: 32
- 被引次数: 0
出版历程
- 收稿日期: 2021-08-03
- 修回日期: 2022-01-18
- 网络出版日期: 2024-04-03
- 刊出日期: 2022-07-06

SAR Image Generation Method via PCGAN for Ship Detection

PAN Lei^1
,,
GUO Yushi^{2
, ,},
LI Hengchao^{3, 4},
WANG Weiye³,
LI Zechen³,
MA Tianyu³

1.
The 10th Research Institute of China Electronics Technology Group Corporation, Chengdu 610036, China
2.
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China
3.
School of Information Science and Technology, Southwest Jiaotong University, Chengdu 611756, China
4.
National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Southwest Jiaotong University, Chengdu 611756, China

摘要

摘要:
针对现有合成孔径雷达（SAR）图像数据生成方法大多无法同时生成舰船图像及其检测标签的问题，面向SAR舰船图像生成及目标检测任务，构建基于位置信息的条件生成对抗网络（PCGAN）. 首先，提出将舰船位置信息作为约束条件用于限制生成图像中舰船的位置，并将其作为舰船图像的检测标签；随后，引入Wasserstein距离稳定PCGAN的训练过程；最后，利用生成的SAR舰船图像及对应检测标签完成YOLOv3网络的端到端训练，实现舰船数据增强与目标检测的协同学习，进而获得更耦合目标检测实际应用的多样性数据. 在HRSID (high resolution SAR image dataset)数据集上的实验结果表明，PCGAN方法能生成清晰、鲁棒的SAR舰船数据，舰船检测准确度最高提升1.01%，验证了所提出方法的有效性.
- 合成孔径雷达 /
- 生成对抗网络 /
- 数据增强 /
- 舰船检测 /
- 位置信息
Abstract:
Most of existing synthetic aperture radar (SAR) image generation methods fail to generate SAR images and their detection labels simultaneously. A position-based conditional generative adversarial network (PCGAN) is constructed for SAR ship image generation and target detection. Firstly, ship position information is used as a constraint and a detection label to restrict its position in the generated image. Then, the Wasserstein distance is further introduced to stabilize the training process of PCGAN. Finally, the generated SAR images and their corresponding labels are applied for the end-to-end training of YOLOv3, so as to realize the cooperative learning of data enhancement and ship detection, and further obtain the diversified ship data more coupled with the practical application of ship detection. Experimental results conducted on HRSID dataset illustrate that PCGAN can generate clear and robust SAR ship data, and the accuracy of ship detection can be improved by up to 1.01%, thus verifying the proposed method.
- synthetic aperture radar /
- generative adversarial network (GAN) /
- data enhancement /
- ship detection /
- position information

HTML全文

图 1 原始生成对抗网络结构

Figure 1. Original GAN structure

下载: 全尺寸图片幻灯片

图 2 SAR图像数据PCGAN生成方法整体框架

Figure 2. Framework of PCGAN based generation method for SAR images

下载: 全尺寸图片幻灯片

图 3 舰船位置信息图像形式转换过程

Figure 3. Conversion process of image form for ship position information

下载: 全尺寸图片幻灯片

图 4 PCGAN的生成器结构

Figure 4. Structure of PCGAN generator

下载: 全尺寸图片幻灯片

图 5 PCGAN的判别器结构

Figure 5. Structure of PCGAN discriminator

下载: 全尺寸图片幻灯片

图 6 HRSID数据集部分图像

Figure 6. Partial images of HRSID

下载: 全尺寸图片幻灯片

图 7 不同尺度/方向舰船目标的转换结果

Figure 7. Conversion results of ship targets with different scales or directions

下载: 全尺寸图片幻灯片

图 8 不同GAN网络的生成结果

Figure 8. Generation results of different GANs

下载: 全尺寸图片幻灯片

表 1 不同尺寸内舰船目标比重

Table 1. Proportion of ship targets in different sizes

尺寸/像素	32 × 32	64 × 64	128 × 128
比例/%	14.18	78.88	99.33

下载: 导出CSV

表 2 不同训练集下的实验结果

Table 2. Experimental results under different training sets %

条件	I/幅
条件	100	200	300
增强前	91.01	92.91	93.11
增强 1.0 倍	91.88	93.61	94.06
增强 1.5 倍	91.79	93.73	94.12
增强 2.0 倍	91.46	93.31	93.93
增强 3.0 倍	91.48	93.33	94.00

下载: 导出CSV

表 3 不同GAN网络的生成图像质量

Table 3. Image quality of different GANs

不同生成网络	FID		SWD
不同生成网络	舰船图像	增强图像	舰船图像	增强图像
GAN	221.9	126.3	3665	2519
WGAN	66.06	96.92	181.7	1692
条件 WGAN	157.1	102.8	448.3	1699
条件 WGAN+方向	99.61	99.07	440.0	1620
PCGAN	86.94	95.62	434.2	1513

下载: 导出CSV

表 4 消融实验结果

Table 4. Results of ablation experiments

不同生成网络	准确度/%
无生成网络	93.11
条件 WGAN	93.71
条件 WGAN+方向	93.88
PCGAN	94.12

下载: 导出CSV

参考文献(23)

[1]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. (2018-04-08)[2020-10-13]. https://arxiv.org/abs/1804.02767.
[2]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[EB/OL]. (2015-12-08)[2020-10-13]. https://arxiv.org/abs/1512.02325.
[3]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[4]	DING J, CHEN B, LIU H W, et al. Convolutional neural network with data augmentation for SAR target recognition[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(3): 364-368.
[5]	朱柳. 高分辨SAR场景分布的重建方法研究[D]. 成都: 电子科技大学,2013.
[6]	刘鹏程. 基于特征调制的SAR欺骗干扰方法研究[D]. 成都: 电子科技大学,2013.
[7]	BALZ T, STILLA U. Hybrid GPU-based single- and double-bounce SAR simulation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(10): 3519-3529.
[8]	HAMMER H, SCHULZ K. Coherent simulation of SAR images[C]//Proceedings of SPIE—The International Society for Optical Engineering 7477. Berlin: [s.n.], 2009: 406-414.
[9]	VAN DEN OORD A, KALCHBRENNER N, KAVUKCUOGLU K. Pixel recurrent neural networks[C]//Proceedings of the 33rd International Conference on International Conference on Machine Learning. New York: [s.n.],2016: 1747-1756.
[10]	KINGMA D P, WELLING M. Auto-encoding variational bayes[EB/OL]. (2013-12-20)[2020-10-13]. https://arxiv.org/abs/1312.6114.
[11]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal:MIT Press, 2014: 2672-2680.
[12]	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL]. (2015-11-19)[2020-10-13]. https://arxiv.org/abs/1511.06434.
[13]	KARRAS T, AILA T, LAINE S, et al. Progressive growing of GANs for improved quality, stability, and variation[EB/OL]. (2017-10-27)[2020-10-13]. https://arxiv.org/abs/1710.10196.
[14]	MIRZA M, OSINDERO S. Conditional generative adversarial nets[EB/OL]. (2014-11-06)[2020-10-13]. https://arxiv.org/abs/1411.1784.
[15]	ARJOVSKY M, BOTTOU L. Towards principled methods for training generative adversarial networks[EB/OL]. (2017-01-17)[2020-10-13]. https://arxiv.org/abs/1701.04862.
[16]	ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[EB/OL]. (2017-01-26)[2020-10-13]. https://arxiv.org/abs/1701.07875.
[17]	徐永士,贲可荣,王天雨,等. DCGAN模型改进与SAR图像生成研究[J]. 计算机科学,2020,47(12): 93-99. doi: 10.11896/jsjkx.200700109 XU Yongshi, BEN Kerong, WANG Tianyu, et al. Study on DCGAN model improvement and SAR images generation[J]. Computer Science, 2020, 47(12): 93-99. doi: 10.11896/jsjkx.200700109
[18]	HUANG H H, ZHANG F, ZHOU Y S, et al. High resolution SAR image synthesis with hierarchical generative adversarial networks[C]//IEEE International Geoscience and Remote Sensing Symposium. Yokohama: IEEE, 2019: 2782-2785.
[19]	GUO J Y, LEI B, DING C B, et al. Synthetic aperture radar image synthesis by using generative adversarial nets[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(7): 1111-1115.
[20]	LU Q L, JIANG H Y, LI G J, et al. Data augmentation method of SAR image dataset based on Wasserstein generative adversarial networks[C]//2019 International Conference on Electronic Engineering and Informatics (EEI). Nanjing: IEEE, 2019: 488-490.
[21]	GAO F, YANG Y, WANG J, et al. A deep convolutional generative adversarial networks (DCGANs)-based semi-supervised method for object recognition in synthetic aperture radar (SAR) images[J]. Remote Sensing, 2018, 10(6): 846.1-846.21.
[22]	WEI S J, ZENG X F, QU Q Z, et al. HRSID: a high-resolution SAR images dataset for ship detection and instance segmentation[J]. IEEE Access, 2020, 8: 120234-120254.
[23]	HEUSEL M, RAMSAUER H, UNTERTHINER T, et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. California: ACM, 2017: 6629-6640.