How computer vision is enabling a circular economy

This post was contributed by Laurent Fabre, Chief Technology Officer and his team from Reezocar, and Ilan Gleiser, Pr. Specialist, Global Impact Computing, AWS and our team.

Computer vision has enormous potential to revolutionize the way we think about sustainability and the circular economy. One of the key ways in which computer vision is already being applied to these areas is using powerful graphics processing units (GPUs) and AWS cloud computing services.

One of the primary applications of computer vision in the circular economy is in product lifecycles. By using advanced computer vision algorithms and machine learning models, manufacturers and supply chain managers can more accurately track and analyze the lifecycle of products, from raw materials sourcing all the way through to end-of-life disposal. This allows companies to better understand the environmental impacts of their products and make more informed decisions about how to design and produce products that are more sustainable and can be easily reused or recycled.

Another area in which computer vision is being applied to the circular economy is in waste reduction. By using advanced image recognition algorithms and computer vision technologies, waste management companies can better sort and categorize recyclable materials, making it easier and more cost-effective to recover and reuse valuable resources.

In addition to these applications, computer vision is also being used to identify opportunities for energy and resource savings, and to monitor and analyze environmental impacts at both a local and global scale. For example, satellite imagery and other forms of remote sensing data can be used to track deforestation, monitor ocean pollution levels, and even create early warning systems to predict the impact of natural disasters on ecosystems.

In this post, we’ll explore how Reezocar is using computer vision to change the way they detect car damage and price used vehicles for re-sale in secondary markets. This technology can be used to determine the useful life of a car and potentially reduce the need for landfill waste, therefore aligning with the goals of the circular economy: designing-out waste and pollution from the environment.

Secondary markets play a critical role in achieving a circular economy by extending the lifespan of products and reducing waste. The benefits of secondary markets go beyond environmental impact, as they also generate economic opportunities, job creation, and community empowerment.

In this context, computer vision is revolutionizing the way we detect car damage. By leveraging the power of machine learning, computer vision can detect even the smallest dents and scratches on a car’s body.

Overall, the applications of computer vision in the circular economy are wide-ranging and growing rapidly. By enabling more efficient and sustainable resource use, computer vision has the power to drive real change in our economy and help create a more sustainable future for us all.

Who is Reezocar?

Reezocar is an online platform for buying and selling used cars in France. It offers customers a safe purchase guarantee and a bespoke shopping help service. With access to over six-million car ads, Reezocar makes it easy to select makes and models that match a buyer’s search criteria. Customers also have the option of purchasing a certified car, with a 15-day money back guarantee included.

Reezocar uses GPU-accelerated machine learning algorithms, convolutional neural networks, and a damage estimation model to calculate car prices. This system helps customers get the best deal on their purchase, as well as an accurate assessment of the vehicle’s condition. The result is a fair price estimate of the car and reassurance that they’re getting a good value for their money.

More than 10,000 customers have already been won over by the Reezocar experience, making it one of the top on-line marketplaces for buying and selling used cars. With its reliable customer service and advanced technology, Reezocar is quickly becoming the go-to destination for those looking to purchase a used car.

Reezocar is committed to its environmental mission and employs various strategies to uphold its climate-conscious values. Here are four key approaches they employ:

1. Extending Vehicle Lifespan: By refurbishing selected vehicles, they have successfully prolonged their lifespan by up to five years, using approximately 5% of their manufacturer’s suggested retail price (MSRP) as refurbishing budget. This initiative has prevented over 30,000 tons of waste from ending up in landfills.
2. Carbon Offset Initiatives: To counterbalance the CO2 emissions produced by these vehicles, they consistently engage in carbon offset practices through their partnership with ReforestAction. Through this collaboration, they have planted 33,000 trees, effectively trapping an estimated 5,000 tons of CO2.
3. Discouraging New Car Production: Recognizing that the most environmentally friendly car is the one that doesn’t need to be built, they actively discourage the production of new cars by prioritizing the refurbishment of used vehicles. This approach has resulted in an estimated avoidance of 40,000 tons of material, considering the increasing weight of modern cars.
4. Efficient Supply Chain: By advocating for a shorter supply chain, they significantly reduce energy consumption during the delivery of vehicles. Additionally, this approach helps them avoid the costs associated with waste management while minimizing the pollution caused by car-related materials such as plastics, thus safeguarding the surrounding ecosystem.
   With a decade of experience and numerous satisfied customers, Reezocar remains dedicated to leading the way in climate action.

The rise of computer vision

For decades, car buyers and dealers have relied on manual inspection to detect body damage on used cars. Until recently, physical inspections were the only way to get a full picture of the car’s condition. Even then, most damage could only be detected with close examination from a trained eye.

The technology leverages a combination of machine learning algorithms and convolutional neural networks to detect dents in vehicles, resulting in a faster, more streamlined, and accurate process.

Initially, the technology employs machine learning algorithms to estimate an initial value of the car based on features such as its model and gearbox type (automatic vs. standard).

Next, Computer vision is used to process the car’s image and identify any form of damage, like scratches, dents, or severe degradation. Any detected damage will be factored into the initial value estimate by subtracting the repair costs. The proposed computer vision system determines the specific parts of the car that are damaged, which helps identify the necessary repairs and their costs.

After computing the repair cost, Reezocar generates a post-repair estimate of the car’s value. This estimate considers factors like age, condition, and make to ensure maximum accuracy. With these tools at their disposal, dealerships can offer more competitive pricing and decide whether a car should be refurbished or sent to the landfill.

Reezocar reference architecture

Figure 1: Reezocar’s architecture schema

Reezocar’s Reference architecture aims to detect car dents using computer vision, as illustrated in Figure 1. Their approach consists of three steps: acquiring a dataset, fine tuning a convolutional neural network, and measuring the success of the model.

First, they acquired and combined a synthetic and a real-world dataset to ensure a diverse and robust sample dataset. Using Amazon EC2 instances, they generated a synthetic dataset by algorithmically deforming 3D CAD models of cars. This helped to create a labeled synthetic dataset of images. Afterwards, they used Amazon SageMaker Ground Truth to annotate a set of real-world images. To achieve this, Reezocar uses Amazon SageMaker Ground Truth to acquire and annotate data, and then employs machine learning models running on GPUs to train a damage estimation model.

By using Amazon SageMaker Ground Truth, Reezocar was able to avoid the need to manage their own data labeling workforce, which would have slowed down innovation and increased costs. This approach allowed Reezocar to focus on their core competencies and optimize their resources for maximum efficiency.

Second, they leveraged GPUs to fine tune a convolutional neural network (CNN) such as Mask R-CNN or Detectron to do object segmentation and damage detection. This CNN model is used to segment car parts and detect car dents.

Reezocar adopted a gradual approach to tackle the dent detection problem. They aimed to ensure that their system could effectively differentiate between damaged and undamaged cars. Once they were satisfied with the performance of the initial model, they submitted images with low confidence for undamaged cars and images of damaged cars to Amazon SageMaker Ground Truth to obtain around 4000 more detailed and accurate annotations, which indicate which parts are damaged. This annotated dataset was then used to train an object detection model that could accurately identify the damaged parts of a car.

Reezocar uses AWS Batch to run inference and augment their data with damage information. Reezocar triggers AWS Batch conditionally by leveraging AWS Step Functions and an Amazon SQS queue. The Step Functions workflow showcased in Reference 1 checks for the arrival of a certain number of events related to the car damage data in the SQS queue. Once the required number of events is present, AWS Step Functions triggers AWS Batch to process the data using the proposed computer vision algorithm on P4d instances. AWS Batch dynamically provisions the optimal quantity and type of compute resources based on the volume and specific resource requirements of the batch jobs submitted.

P4d instances are a specific type of Amazon EC2 instance family that is optimized for high-performance computing and includes powerful NVIDIA GPUs. These instances provide the necessary computational power for running machine learning models and computer vision algorithms efficiently, making them suitable for Reezocar’s car damage detection system.

This combination of Step Functions, SQS, and AWS Batch allows Reezocar to efficiently process the data.

Evaluating the model is in the eye of the beholder

Finally, to evaluate the performance of their dent detection models, Reezocar used a combination of accuracy and mean Average Precision (mAP) metrics. The accuracy metric was used to assess the classifier’s performance, while the mAP metric was used to evaluate the object detection model’s performance. They set a target accuracy of 0.85 and a target mAP of 0.9 to ensure high levels of accuracy and reliability in their results. These metrics allowed Reezocar to measure the success of their object detection model and assess the quality of the system’s output. The process is illustrated in Figure 2.

Figure 2: The damage detection system developed by Reezocar combines the generation of a synthetic dataset with real-world data labeling to ensure a diverse and robust dataset. The system is designed to detect relevant images that require labeling, which contributes to the overall efficiency and effectiveness of the damage detection system.

Reezocar’s objective is to determine the refurbishing cost of cars, which involves an additional step to detect the severity of the damage in parts. The severity assessment process is illustrated in Figure 3, and the severity estimation is used to determine the repair cost of the damaged part. Table 1 shows that the repair cost estimation method involves applying a percentage of the initial price of the part, which varies according to the severity of the damage. In other words, different percentages are used based on the severity of the damage to estimate the repair cost accurately.

Figure 3: Damage severity assessment pipeline. The severity assessment is based on the normal consistency of the damaged part.

Table 1: Repair cost. Reezocar uses a percentage of the initial price of the damaged part to estimate the repair cost. The repair cost estimation process involves applying different percentages based on the severity of the damage.

Through the implementation of their damage detection pipeline, Reezocar has achieved a high degree of automation with a precision rate of 86.7%. However, due to challenging light conditions and reflective materials, some cars are being mislabeled as damaged. Reezocar has identified a mislabeling rate of 15.7%, and are currently labeling data in order to retrain the damage detection model.

The team was able to achieve a notable performance improvement by using Amazon Sagemaker Ground Truth to obtain 4000 labeled images, resulting in an increase in their mean Average Precision (MAP) from 0.22 to 0.3. This success has validated the effectiveness of their data-labeling pipeline and has motivated the company to pursue further data acquisition for labeling to enhance the performance of their system. As part of this effort, Reezocar is actively collecting images of damaged cars and labeling them to reach their goal of a MAP of 0.9.

In the figures below, we see some qualitative results of the proposed damage detection model. For example, in Figure 4, an image of a car with damaged parts is displayed, and the proposed model accurately detects and segments the damaged parts, as shown in the second row, first column. Additionally, they used a car-part segmentation model (See Row 1; Column 3 of Figure 4) to identify which specific part is damaged.

Figure 4: example of a wrecked car.

Figure 5 shows an image of an undamaged car, while Figure 6 showcases the performance of the proposed method on scratched parts of a car, providing a closer look at the results.

Figure 5: Example of an undamaged car.

Figure 6: Example of a car with a scratched part.

Conclusion

The use of computer vision to detect car dents is quickly becoming a game-changer in the automotive industry. Machine learning and convolutional neural networks are allowing for a more accurate detection of car dents than ever before, leading to improved repair and maintenance processes.

Companies like Reezocar are leveraging computer vision-based damage estimation models to accurately and efficiently calculate car prices. Thanks to these models, Reezocar is now able to calculate the car price with a Mean Absolute Percentage (MAP) error of just 6%, improving the accuracy and efficiency of their pricing process.

This technology is not only helping to extend the life of cars but also to reduce landfill waste. It’s also helping to make the car buying process simpler and more transparent. As computer vision technology continues to improve, the way we detect and repair car dents will likely change as well.

By combining the power of Amazon EC2 instances, NVIDIA GPUs, AWS Batch linked by Elastic Fabric Adapter (EFA), and machine learning powered computer vision models, Reezocar can accurately detect car dents, ultimately extending the lifespan of selected cars by up to 5 years. By refurbishing these cars and reselling them in the secondary market, Reezocar helps reduce landfill waste and promote circularity.