W. Giegerich, T.S. Porter, K. Shuttleworth, L. Forte III, Ph.J. Schneider, K.W. Oh
University at Buffalo,
United States
Keywords: vehicle dashboard, warning light detection, warning light classification, machine learning, classification, detection
Summary:
This research focuses on the development and implementation of a machine-learning model that detects and classifies warning lights present on a vehicle's dashboard. A YOLOv5 object detection model was trained on photos of vehicle dashboards and output bounding box predictions of warning lights present in each image selected from among 18 different classes (Figure 1). An image dataset of vehicle dashboards with and without warning lights present was curated and labeled to train, validate, and test the model. The model achieved an overall macro-average F1 score of 0.91 amongst its top 15 performing classes. Prior Work: The vehicle's dashboard warning light system is one of the fastest and most effective methods to identify systems or components that require maintenance on a vehicle. It provides key insight into the state and overall condition of a vehicle and helps to ensure the safety of drivers and furthermore, helps buyers determine vehicle value. For these reasons it is crucial for drivers to understand the meaning of the dashboard lights present in their vehicle [1]. ACV Auctions, an online wholesale automotive marketplace, has taken note of this. ACV takes a photo of the dashboard in every vehicle they sell and provides it to their customers. Moreover, they work to identify any warning lights that are present along with their importance. Automatic detection of dashboard warning lights utilizing a machine-learning model however, reduces the possibility of human error. Ensuring those present are detected and their meaning / importance is properly disclosed. Methods: With the use of a mobile phone, ACV Auctions has taken dashboard photos of over a million vehicles consisting of hundreds of different vehicle makes, models, and years. Through investigation and research of the different warning lights present within these dashboards, 18 image classes were created to classify all lights used by manufacturers across the automotive industry. Utilizing the Computer Vision Annotation Tool (CVAT) over 10,000 dashboard photos with and without lights present were labeled to train a YOLOv5 machine learning model across the 18 image classes defined in Table 1. Table 2 shows the datasets used to train the model. Experiments & Results: To test the performance of the ML model, a test dataset was created (Table 1). The test dataset consisted of over 1,200 dashboard images and over 2,000 individual warning light labels. The test resulted with the model achieving an overall prediction macro-average F1 score of 0.91 across the top 15 performing classes. With some classes having low amounts of training and testing data, more dashboard photos with lights from those classes will be labeled and used in the future to further train / test the model. In conclusion, the system developed in this work provides its users the ability to quickly detect and classify dashboard warning lights present on vehicles from across the automotive industry, providing a needed understanding of inspected vehicles’ conditions.