Open the Black Box – Visualising CNN to Understand Its Decisions on Road Network Performance Level

Junxian Li; Zhizhou Wu; Zhoubiao Shen

doi:10.7307/ptt.v34i4.4037

Authors

Junxian Li Tongji University
Zhizhou Wu Tongji University
Zhoubiao Shen Shanghai Urban Construction Design & Research Institute (Group) Co., Ltd.

DOI:

https://doi.org/10.7307/ptt.v34i4.4037

Keywords:

visualisation, convolutional neural network (CNN), gradient weighted class activation mapping (Grad-CAM), pretrained network, road network performance

Abstract

Visualisation helps explain the operating mechanisms of deep learning models, but its applications are rarely seen in traffic analysis. This paper employs a convolu-tional neural network (CNN) to evaluate road network performance level (NPL) and visualises the model to en-lighten how it works. A dataset of an urban road network covering a whole year is used to produce performance maps to train a CNN. In this process, a pretrained network is introduced to overcome the common issue of inadequa-cy of data in transportation research. Gradient weighted class activation mapping (Grad-CAM) is applied to vi-sualise the CNN, and four visualisation experiments are conducted. The results illustrate that the CNN focuses on different areas when it identifies the road network as dif-ferent NPLs, implying which region contributes the most to the deteriorating performance. There are particular visual patterns when the road network transits from one NPL to another, which may help performance prediction. Misclassified samples are analysed to determine how the CNN fails to make the right decisions, exposing the model’s deficiencies. The results indicate visualisation’s potential to contribute to comprehensive management strategies and effective model improvement.

References

Zhang J, et al. Data-driven intelligent transportation systems: A survey. IEEE Transactions on Intelligent Transportation Systems. 2011;12(4): 1624-1639. doi: 10.1109/TITS.2011.2158001.

Pamula T. Road traffic conditions classification based on multilevel filtering of image content using convolutional neural networks. IEEE Intelligent Transportation Systems Magazine. 2018;10(3): 11-21. doi: 10.1109/MITS.2018.2842040.

Duan Y, Lv Y, Liu Y, Wang F. An efficient realization of deep learning for traffic data imputation. Transportation Research Part C - Emerging Technologies. 2016;72: 168-181. doi: 10.1016/j.trc.2016.09.015.

Zhao Z, et al. LSTM network: A deep learning approach for short-term traffic forecast. IET Intelligent Transport Systems. 2017;11(3): 68-75. doi: 10.1049/iet-its.2016.0208.

Cheng Z, Wang W, Lu J, Xing X. Classifying the traffic state of urban expressways: A machine-learning approach. Transportation Research Part A - Policy and Practice. 2020;137: 411-428. doi: 10.1016/j.tra.2018.10.035.

Hoang N, Le-Minh K, Tao W, Chen C. Deep learning methods in transportation domain: A review. IET Intelligent Transport Systems. 2018;12(9): 998-1004. doi: 10.1049/iet-its.2018.0064.

Karlaftis MG, Vlahogianni EI. Statistical methods versus neural networks in transportation research: Differences, similarities and some insights. Transportation Research Part C - Emerging Technologies. 2011;19(3): 387-399. doi: 10.1016/j.trc.2010.10.004.

Selvaraju RR, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision. 2020;128(2): 336-359. doi: 10.1007/s11263-019-01228-7.

Du S, Li T, Gong X, Horng S. A hybrid method for traffic flow forecasting using multimodal deep learning. International Journal of Computational Intelligence Systems. 2020;13(1): 85-97. doi: 10.2991/ijcis.d.200120.001.

Bogaerts T, et al. A graph CNN-LSTM neural network for short and long-term traffic forecasting based on trajectory data. Transportation Research Part C - Emerging Technologies. 2020;112: 62-77. doi: 10.1016/j.trc.2020.01.010.

Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. Proceedings of ECCV, 6-12 Sep. 2014, Zurich, Switzerland. Cham, Switzerland: Springer International Publishing; 2014. p. 818-833. doi: 10.1007/978-3-319-10590-1_53.

Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M. Striving for simplicity: The all convolutional net. Proceedings of ICLR, 7-9 May 2015, San Diego, CA, USA. 2015. arXiv:1412.6806.

Mahendran A, Vedaldi A. Visualizing deep convolutional neural networks using natural pre-images. International Journal of Computer Vision. 2016;120(3): 233-255. doi: 10.1007/s11263-016-0911-8.

Dosovitskiy A, Brox T. Inverting visual representations with convolutional networks. Proceedings of IEEE Conf. on CVPR, 27-30 June 2016, Las Vegas, NV, USA. Los Alamitos, CA, USA: IEEE Computer Society; 2016. p. 4829-4837. doi: 10.1109/CVPR.2016.522.

Zhou B, et al. Learning deep features for discriminative localization. Proceedings of IEEE Conf. on CVPR, 27-30 June 2016, Las Vegas, NV, USA. Los Alamitos, CA, USA: IEEE Computer Society; 2016. p. 2921-2929. doi: 10.1109/CVPR.2016.319.

Lin M, Chen Q, Yan S. Network in network. Proceedings of ICLR, 14-16 Apr. 2014, Banff, Canada. 2014. arXiv:1312.4400, 2014.

Simonyan K, Zisserman A. Very deep convolutional networks for large scale image recognition. Proceedings of ICLR, 7-9 May 2015, San Diego, CA, USA. 2015. arXiv:1409.1556.

Jogin M, et al. Feature extraction using convolution neural networks (CNN) and deep learning. Proceedings of IEEE Int. Conf. on RTEICT, 18-19 May 2018, Bangalore, India. Piscataway, NJ, USA: IEEE; 2018. p. 2319-2323. doi: 10.1109/RTEICT42901.2018.9012507.

Agrawal P, Girshick R, Malik J. Analyzing the performance of multilayer neural networks for object recognition. Proceedings of ECCV, 6-12 Sept. 2014, Zurich, Switzerland. Cham, Switzerland: Springer International Publishing; 2014. p. 329-344. doi: 10.1007/978-3-319-10584-022.

Chollet F. Deep Learning with Python. Shelter Island, NY, USA: Manning Publications; 2017.

Hinton G. Coursera Course Lectures. 2012. http://www.cs.toronto.edu/~hinton/coursera_slides.html [Accessed 21st Dec. 2021].

Bach S, et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE. 2015;10(7): e0130140. doi: 10.1371/journal.pone.0130140.

Shrikumar A, Greenside P, Kundaje A. Learning important features through propagating activation differences. 2019. arXiv:1704.02685v2.

Simonyan K, Vedaldi A, Zisserman A. Deep inside convolutional networks: Visualising image classification models and saliency maps. Proceedings of ICLR, 14-16 Apr. 2014, Banff, AB, Canada. 2015. arXiv: 1312.6034.