Automatic Traffic State Recognition from Road Videos Based on 3D Convolution Neural Network

In order to directlyextract effective traffic information from videos, a traffic state recognition method based on 3D CNN (3D convolutional neural networks)was put forward. Firstly, with the deep convolutional network C3D (convolutional 3D) as 3D CNN prototype, the number and position of convolutional layers, convolutional kernelsize and 3D convolutional depth were optimized and adjusted; thus 37 candidate models were built. Secondly, video datasets were established to systematically train and test candidate models, and a traffic state recognition model C3D* was proposed. Then, tests and analysis were conducted on traffic state recognition results of C3D* and existing 3D convolutional models. At last, traffic recognition results were compared between C3D* and commonly used 2D convolutional networks. The results show that for video traffic state recognition, the average F value of C3D* reaches 91.32%, which is 12.24%, 26.72% and 28.02% higher than that of C3D, R3D (region convolutional 3D network) and R(2+1)D (resnets adopting 2D spatial convolution and a 1D temporal convolution), respectively, demonstrating that the proposed model C3D* is more accurate and effective. Compared with image recognition results from LeNet, AlexNet, GoogleNet and VGG16, the average C3D* is 32.61%, 69.91%, 50.11% and 69.17% higher respectively, proving that 3D video convolution F value of outperforms 2D image convolution in terms of traffic status recognition.