W.V. Giegerich, A. Stone, L. Forte III, P.J. Schneider, K.W. Oh
University at Buffalo,
United States
Keywords: automotive imaging, image classification, data quality
Summary:
This research focuses on the development of a vehicle undercarriage image quality machine learning model. The model returns the quality of the image across several categories which are used for classifying image defects. Through analysis of 1M+ images taken with a proprietary Virtual Lift™ imaging system [1], a list of image issues, defects, and causes were identified and classified. An image dataset incorporating all issues / defects was then created and labeled for training and validating a machine learning neural network. The neural network, a ResNet 34, classified the undercarriage images with an average RoC area under the curve of 0.90. An ensemble was created from both the image quality model and an object detection model to increase overall performance of the object detection model. Applications expand to driving image data quality upstream at the time of image capture by providing real time feedback to the end user and as a second check to results from an automated imaging platform. Prior Work: The undercarriage of a vehicle is a significant indicator of its purchase price, condition, safety, and future longevity [2]. It can provide insights on upgrades or problems associated with the vehicle such as an enhanced suspension or improper repairs, damaged / missing components, or major rust. For proper assessment, a high quality photo which includes adequate lighting, high resolution, and full end-to-end display of the undercarriage is required. Methods: In this work, the use of an imaging hardware system that utilizes a mobile camera pointed at a reflective surface was used to image the underside of a vehicle [1]. Through human operating error, images which contain defects or issues are often generated preventing proper assessment of the vehicle’s undercarriage. In an attempt to push image data quality upstream and acquire more usable photos for vehicle assessment, classification of the resulting images and their defects was conducted. Eight main defects were identified and categorized. Figure 1 provides an overview of the defects identified resulting from user error during the image collection process. Utilizing the Computer Vision Annotation Tool (CVAT), 1100 undercarriage images were labeled with the corresponding eight tags (Figure 2). With a labeled dataset, a machine learning model was trained to identify if an image was usable (defect free) for proper undercarriage assessment and to identify the root cause of the problem. Experiments & Results: To test the performance of our model, we created a validation set of 220 images with and without defects. Running the model on these datasets, the model resulted in identifying the correct type of image at an average RoC curve area score of 0.90 (Figure 3). After analyzing over 10,000 images, we identified causes for user error which will allow for the ability to fix the cause of the seen defects, furthermore improving the ability of our users to better assess vehicle condition. This model has the potential to be integrated back into the image collection process providing direct user feedback and suggestions to improve data capture quality.