Enhancing Autonomous Vehicle Navigation by Detecting Lane and Objects based on LaneNet and CustomYOLOv5

Jayamani SIDDAIYAN; Kumar PONNUSAMY

doi:10.7307/ptt.v37i3.669

Authors

Jayamani SIDDAIYAN K.S. Rangasamy College of Technology
Kumar PONNUSAMY K.S. Rangasamy College of Technology

DOI:

https://doi.org/10.7307/ptt.v37i3.669

Keywords:

drivable space detection, intelligent vehicle, LaneNet, YOLO

Abstract

Lane and object detection are the major concerns of an autonomous vehicle’s ability to move continuously without creating any traffic congestion or collisions. Highly populated rural and urban roads are still facing many challenges to enabling the intelligent transport system with an end-to-end customer connectivity. The proposed work is to identify the drivable space by combining lane line detection by using the LaneNet with sliding window and front road object detection and using the customised YOLOv5. The appropriate pre-processing methods are carried out to reduce the computational complexity and speed up the process. Followed by pre-processing, the reference line is assumed at the far-end distance from the host vehicle to identify the driving space. The lane line borders and objects bounding box coordinate intersecting points on the reference line are picked up to calculate the drivable space. Finally, the proposed system is validated on various public and own datasets. Lane line detection and object detection accuracy of 97% and 98%, respectively are achieved by the LaneNet with sliding windows and custom YOLOv5.

References

McCall JC, Trivedi MM. Video-based lane estimation and tracking for driver assistance: Survey, system, and evaluation. IEEE Transactions on Intelligent Transportation Systems. 2006;7(1):20–37. DOI: 10.1109/tits.2006.869595.

Zheng T, et al. CLRNet: Cross layer refinement network for lane detection. ArXiv (Cornell University). 2022. Available from: https://arxiv.org/abs/2203.10350.

Narayan A, Tuci E, Labrosse F, Alkilabi MHM. A dynamic colour perception system for autonomous robot navigation on unmarked roads. Neurocomputing. 2017;275:2251–63. DOI: 10.1016/j.neucom.2017.11.008.

Yang F, et al. Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Transactions on Intelligent Transportation Systems. 2019;21(4):1525–35. DOI: 10.1109/tits.2019.2910595.

Zürn J, Burgard W, Valada A. Self-supervised visual terrain classification from unsupervised acoustic feature learning. ArXiv (Cornell University). 2019. Available from: https://arxiv.org/abs/1912.03227.

Lee C, Moon JH. Robust lane detection and tracking for real-time applications. IEEE Transactions on Intelligent Transportation Systems. 2018;19(12):4043–8. DOI: 10.1109/tits.2018.2791572.

Tian Y, et al. Lane marking detection via deep convolutional neural network. Neurocomputing. 2017;280:46–55. DOI: 10.1016/j.neucom.2017.09.098.

Yu Y, Xu M, Gu J. Vision‐based traffic accident detection using sparse spatio‐temporal features and weighted extreme learning machine. IET Intelligent Transport Systems. 2019;13(9):1417–28. DOI: 10.1049/iet-its.2018.5409.

Qiu Z, Zhao J, Sun S. MFIALane: Multiscale feature information aggregator network for lane detection. IEEE Transactions on Intelligent Transportation Systems. 2022;23(12):24263–75. DOI: 10.1109/tits.2022.3195742.

Ahangar MN, Ahmed QZ, Khan FA, Hafeez M. A survey of autonomous vehicles: Enabling communication technologies and challenges. Sensors. 2021;21(3):706. DOI: 10.3390/s21030706.

Huang Y, Chen Y. Survey of state-of-art autonomous driving technologies with deep learning. 20th International Conference on Software Quality. 2020;221–8. DOI: 10.1109/qrs-c51114.2020.00045.

Garcia-Garcia A, et al. A survey on deep learning techniques for image and video semantic segmentation. Applied Soft Computing. 2018;70:41–65. DOI: 10.1016/j.asoc.2018.05.018.

Minaee S, et al. Image segmentation using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2021;1. DOI: 10.1109/tpami.2021.3059968.

Haris M, Hou J. Obstacle detection and safely navigate the autonomous vehicle from unexpected obstacles on the driving lane. Sensors. 2020;20(17):4719. DOI: 10.3390/s20174719.

Haris M, Glowacz A. Lane line detection based on object feature distillation. Electronics. 2021;10(9):1102. DOI: 10.3390/electronics10091102.

Huang DY, et al. Vehicle detection and inter-vehicle distance estimation using single-lens video camera on urban/suburb roads. Journal of Visual Communication and Image Representation. 2017;46:250–9. DOI: 10.1016/j.jvcir.2017.04.006.

Yuan Y, Jiang Z, Wang Q. Video-based road detection via onli.ne structural learning. Neurocomputing. 2015;168:336–47. DOI: 10.1016/j.neucom.2015.05.092.

Conrad P, Foedisch M. Performance evaluation of color based road detection using neural nets and support vector machines. 32nd Applied Imagery Pattern Recognition Workshop, 2003 Proceedings. 2004;157–60. DOI: 10.1109/aipr.2003.1284265.

Bertozzi M, et al. Artificial vision in road vehicles. Proceedings of the IEEE. 2002;90(7):1258–71. DOI: 10.1109/jproc.2002.801444.

Shang E, et al. Robust unstructured road detection: The importance of contextual information. International Journal of Advanced Robotic Systems. 2013;10(3):179. DOI: 10.5772/55560.

Yan C, et al. A highly parallel framework for HEVC coding unit partitioning tree decision on many-core processors. IEEE Signal Processing Letters. 2014;21(5):573–6. DOI: 10.1109/lsp.2014.2310494.

Le TT, Lin CY, Piedad EJ. Deep learning for noninvasive classification of clustered horticultural crops – A case for banana fruit tiers. Postharvest Biology and Technology. 2019;156:110922. DOI: 10.1016/j.postharvbio.2019.05.023.

Huang R, et al. A rapid recognition method for electronic components based on the improved YOLO-V3 network. Electronics. 2019;8(8):825. DOI: 10.3390/electronics8080825.

Min W, et al. New approach to vehicle license plate location based on new model YOLO‐L and plate pre‐identification. IET Image Processing. 2019;13(7):1041–9. DOI: 10.1049/iet-ipr.2018.6449.

Cao CY, et al. Investigation of a promoted you only look once algorithm and its application in traffic flow monitoring. Applied Sciences. 2019;9(17):3619. DOI: 10.3390/app9173619.

Junos MH, Khairuddin ASM, Thannirmalai S, Dahari M. An optimized YOLO‐based object detection model for crop harvesting system. IET Image Processing. 2021;15(9):2112–25. DOI: 10.1049/ipr2.12181.

Huang Y, Li Y, Hu X, Ci W. Lane detection based on inverse perspective transformation and Kalman filter. KSII Transactions on Internet and Information Systems. 2018;12(2). DOI: 10.3837/tiis.2018.02.006.

Bonin-Font F, Burguera A, Ortiz A, Oliver G. Concurrent visual navigation and localisation using inverse perspective transformation. Electronics Letters. 2012;48(5):264. DOI: 10.1049/el.2011.3577.

Neven D, et al. Towards end-to-end lane detection: An instance segmentation approach. 2022 IEEE Intelligent Vehicles Symposium (IV). 2018;286–91. DOI: 10.1109/ivs.2018.8500547.

Ma N, Pang G, Shi X, Zhai Y. An all-weather lane detection system based on simulation interaction platform. IEEE Access. 2018;8:46121–30. DOI: 10.1109/access.2018.2885568.

Park KY, Hwang SY. An improved Haar-like feature for efficient object detection. Pattern Recognition Letters. 2014;42:148–53. DOI: 10.1016/j.patrec.2014.02.015.

Arróspide J, Salgado L, Camplani M. Image-based on-road vehicle detection using cost-effective histograms of oriented gradients. Journal of Visual Communication and Image Representation. 2013;24(7):1182–90. DOI: 10.1016/j.jvcir.2013.08.001.

Aziz L, Salam MdSBH, Sheikh UU, Ayub S. Exploring deep learning-based architecture, strategies, applications and current trends in generic object detection: A comprehensive review. IEEE Access. 2020;8:170461–95. DOI: 10.1109/access.2020.3021508.

Park JJ, Pan Y, Yi G, Loia V. Advances in computer science and ubiquitous computing. Lecture notes in electrical engineering. 2016. DOI: 10.1007/978-981-10-3023-9.

Xu B, et al. Automated cattle counting using Mask R-CNN in quadcopter vision system. Computers and Electronics in Agriculture. 2020;171:105300. DOI: 10.1016/j.compag.2020.105300.

Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016;779–88. DOI: 10.1109/cvpr.2016.91.

Zhou F, Zhao H, Nie Z. Safety helmet detection based on YOLOv5. 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA). 2021;6–11. DOI: 10.1109/icpeca51329.2021.9362711.

Lee DH, Liu JL. End-to-end deep learning of lane detection and path prediction for real-time autonomous driving. Signal Image and Video Processing. 2022;17(1):199–205. DOI: 10.1007/s11760-022-02222-2.

Haghighat A, Sharma A. A computer vision‐based deep learning model to detect wrong‐way driving using pan–tilt–zoom traffic cameras. Computer-Aided Civil and Infrastructure Engineering. 2022;38(1):119–32. DOI: 10.1111/mice.12819.

Li Z, et al. Lane line detection network based on strong feature extraction from USFDNet. 2024 IEEE 4th International Conference on Power, Electronics and Computer Applications (ICPECA). 2024. DOI: 10.1109/icpeca60615.2024.10470951.

Kumar S, Pandey A, Varshney S. Exploring the impact of deep learning models on lane detection through semantic segmentation. SN Computer Science. 2024;5(1). DOI: 10.1007/s42979-023-02328-5.

Cao Y, Li C, Peng Y, Ru H. MCS-YOLO: A multiscale object detection method for autonomous driving road environment recognition. IEEE Access. 2023;11:22342–54. DOI: 10.1109/access.2023.3252021.

Ji SJ, Ling QH, Han F. An improved algorithm for small object detection based on YOLO v4 and multi-scale contextual information. Computers & Electrical Engineering. 2022;105:108490. DOI: 10.1016/j.compeleceng.2022.108490.

Majumder M, Wilmot C. Automated vehicle counting from pre-recorded video using you only look once (YOLO) object detection model. Journal of Imaging. 2023;9(7):131. DOI: 10.3390/jimaging9070131

Song Y, et al. MEB-YOLO: An efficient vehicle detection method in complex traffic road scenes. Computers, Materials & Continua/Computers, Materials & Continua. 2023;75(3):5761–84. DOI: 10.32604/cmc.2023.038910.

Lv Z, et al. Road scene multi-object detection algorithm based on CMS-YOLO. IEEE Access. 2023;11:121190–201. DOI: 10.1109/access.2023.3327735.