AWS Partner Network (APN) Blog
Unlocking the Power of AWS for Accelerated AI/ML Enablement: Eviden’s DeepRacer Journey
By Mark Ross, Global Chief AWS Architect – Eviden
Sathya Paduchuri, Partner Solutions Architect – AWS
 |
| Eviden |
 |
Eviden, a next-gen technology leader
Eviden, a next-generation technology leader in data-driven, trusted, and sustainable digital transformation, is proud to share its journey with the AWS DeepRacer platform. As an AWS Premier Partner, Eviden leverages the power of Amazon Web Services (AWS) to drive innovation and empower its global team of 47,000 world-class talents. In this post, we’ll explore how Eviden harnessed the AWS DeepRacer platform to accelerate its employees’ AI/ML learning and development, fostering a culture of continuous improvement and community engagement.
Eviden recognized the potential of the AWS DeepRacer platform to complement its existing training and certification programs, offering employees hands-on experience in areas such as reward function strategy, hyperparameter tuning, and Python programming. To further enrich the learning experience, Eviden introduced several innovative initiatives, including a global league to encourage employees to compete and learn from one another. As well as providing racers access to the DeepRacer open-source community project, DeepRacer on the Spot (DOTS), which exposed them to the use of Amazon Elastic Compute Cloud(EC2) Spot instances, Amazon Simple Storage Service (Amazon S3) object storage, and cost optimization strategies.
Empowering AI/ML Learning through AWS DeepRacer
AWS DeepRacer is a 1/18th scale autonomous vehicle powered by reinforcement learning, providing an engaging way for individuals to dive into the world of AI and machine learning. Racers create a reward function to incentivize car behaviour in a virtual simulator, tuning hyperparameters and adjusting the action space (speed and steering angles) to optimize performance.
The AWS DeepRacer platform provides a fun and engaging way to get into AI and machine learning. The training process is analogous to training a dog, where racers program rewards for desired behaviour and penalties for undesirable behaviour, rather than programming absolute behaviours like turning left or right. For example, racers can reward the vehicle for staying near the centreline or penalize it for going off-track.
Over time, the model learns to perform actions that are more rewarding more often and avoid actions that are less rewarding or penalized. This iterative process allows the model to develop an understanding of the desired behaviour without explicit programming for every scenario.
Using DeepRacer as a vehicle for AI/ML Learning
Figure 1– Eviden’s approach to using deepracer to enhance developer skills
Exposure to the AWS DeepRacer platform provides racers the opportunity to learn about reward function strategy, hyperparameter tuning, and Python programming. However, Eviden recognized the potential to expand the learning outcomes beyond these core concepts. To achieve this, Eviden launched a company-wide global league with a focus on both racing and continuous learning.
To complement the AWS DeepRacer console experience, Eviden, offered racers access to DeepRacer open-source community projects, for example DeepRacer on the Spot (DOTS). DOTS provides racers an opportunity to use AWS CloudShell or a local IDE to deploy their AWS CloudFormation stack, using EC2 Spot instances to develop their models. Not only is this a cost-effective method, but it exposes racers to spot instances, both in terms of their nature like frequency of spot interruption. Market-based pricing that varies by region, Amazon S3 object storage to frequently store their models in case of interruption, and tools such as git. Racers were also given a budget, so cost discipline and setting up billing alarms was a learning point too.
Figure 2 – Architecture diagram to ensure developer cost discipline during model development
In addition to model training, racers were challenged to perform analysis to improve their models through ‘Data Driver Racing’ (D2R). Racers used a combination of tools and techniques to extract data from their training to make data-driven decisions about who their models could be improved.
Jupyter Notebooks were used to perform log analysis, utilizing several Python frameworks: NumPy, pandas and plotly. Many racers signed up to the Sagemaker Studio Labs, which provides time-bound daily access to CPU or GPU instances for machine learning and data analysis tasks. Racers were able to plot the track waypoints, heatmaps of training rewards, plot lap times to see if models were improving as well as understand at which points of the track they gained the largest reward. Racers also had access to live Grafana dashboards during training to understand in near real time how their model was performing during the training process. Using domain knowledge of the task in hand, racers then exploited this data to make changes. Either to their reward function, hyperparameters or action space, to decide when sufficient training had occurred, or when to abandon certain training as results were not improving.
Figure 3 – Eviden’s approach to data-driven racing as part of deepracer league 2024
Generative AI Enhances Model Evaluation and Improvement
Racers were also provided access to AWS Generative AI solutions powered by Amazon Bedrock and PartyRock, an Amazon Bedrock Playground. An initial PartyRock application ‘AWS DeepRacer Optimiser’ was made available to suggest improvements to their models if they provided their reward function, hyperparameters, and action space. Racers could then remix the PartyRock application to make further refinements themselves by enhancing the prompt engineering.
Participants utilized an Amazon Bedrock powered workshop to deploy an agent and a DeepRacer specific Retrieval Augmented Generation (RAG) vector store to augment the out-of-the-box Amazon Titan and Anthropic Large Language Model (LLM) with domain-specific knowledge. Racers could then have a conversation with the agent about their models and how they might improve them.
Figure 4 – Racers leveraging Bedrock with Amazon titan and Anthropic LLMs to improve their models
Sim2Real and Machine Learning overfit and underfit models
Machine learning relies on training data, and in the case of DeepRacer this involves learning to navigate a virtual racetrack. However, the virtual environment presents a challenge – each lap, the autonomous vehicle encounters the same surroundings without variation, except for the changes caused by its position on the track. This problem exists in the real world and is an important aspect of machine learning to understand and be able to compensate for.
Machine learning models can struggle to generalize beyond the specific scenarios and conditions present in their training data. A model trained solely on certain types of manufacturing faults may fail to recognize faults it has not encountered before, especially in components or contexts outside the scope of the training dataset. Similarly, a healthcare machine learning model trained on data depicting a condition in a specific part of the body may not be able to identify the same condition in a different part of the body. In the context of DeepRacer this challenge can manifest itself in models that perform well in the virtual world but struggle to complete laps around the physical track. The virtual world used for training data lacks various real-world factors that the model has not encountered. Such as variations in lighting conditions, the presence of people or objects beyond the track barriers, and surface irregularities or wrinkles on the physical track.
Given that racers had no access to test their models in the real-world before physical racing, evaluating whether their models were well-generalized posed a significant challenge. To address this, Eviden leveraged Stable Diffusion, to generate new images from the original training data that the models had never encountered before. Analysis could be performed to see what features of the images the model was using to make decisions, as well as the probability distribution of which action (combination of speed and steering angle) the model would take based on a given image. This informed the racer on the likelihood of the model being well generalized and likely to perform in the physical race, rather than being overfit to the training data and unlikely to perform well in the physical race. In the image below, on the left, the well-generalized model effectively identifies and focuses on relevant track features, exhibiting a confident and concentrated probability distribution of actions tailored to the presented scenario. In contrast, the image on the right depicts a poorly generalized model that struggles to identify meaningful features and exhibits a wide distribution of actions it may take.
Figure 5– Racers using Bedrock with Stable Diffusion
Conclusion
AWS DeepRacer is an excellent resource to give people hands on access to AWS, providing opportunities for people to learn in a fun, community environment. The competitive nature of hosting an event drives participants to want to improve their models. This leads to learning outcomes across a range of transferable skills important to the developer, engineer, data engineer personas and beyond, as well as access to cutting-edge technology like Generative AI.
Physical events are a real spectacle, capable of bringing together teams from disparate parts of an organization in a friendly, but competitive, way to drive community and wellbeing initiatives through having fun together. There’s an AWS public league to join for those who want to take their journey to the next level and compete against the ‘pro’ racers, start your engines.
.
Eviden – AWS Partner Spotlight
Eviden is an AWS Premier Tier Services Partner and MSP with many AWS Competencies including Security and Level 1 Managed Security Services Provider (MSSP) Consulting.