Scaling ML Education With AWS DeepRacer

In this blog post I will outline how we used DeepRacer to scale Machine Learning (ML) education across our organisation.

The blog will cover:

• How to acquire the cars, track and build the associated event space.

• The various roles on our Pit Crew and how each contributes to the success of the event.

• How AWS can support you in running your event along with links to training resources.

• The timeline for events leading up to our championship race and how to keep your team engaged.

• The costs for the event and the results we saw for the investment.

Introduction

AWS DeepRacer is a service offered by Amazon Web Services (AWS) that combines machine learning, cloud computing, and robotics to provide a platform for learning and experimenting with reinforcement learning.

Reinforcement learning is a type of machine learning where agents learn to make decisions by interacting with an environment and receiving feedback in the form of rewards.

In the context of DeepRacer, the environment is a physical or virtual racetrack, and the agent is an autonomous racing car. Participants can train and develop the car's racing skills using reinforcement learning techniques, allowing the car to learn how to navigate the track and optimize its racing performance over time.

Overall, AWS DeepRacer is an educational and hands-on way for people to dive into reinforcement learning and gain practical experience in training AI models to perform specific tasks, such as autonomous racing.

Between April 2023 and July 2023, I put together a team to organise and run a DeepRacer event at CDL with the aim to scale Machine Learning education across our organisation.

Equipment Required

The Track

We used the 2018 DeepRacer track. This is the smallest track available, and it provided us with some flexibility on location. The track is now referred to as the A-Z Speedway.

The spec for the track is:

• Track size: 26’ x 17’ or 7.9248m x 5.1816m

The track should have the following colours:

• Field ("the green") = PMS 3395C
• Road surface and AWS logo = PMS 432C
• Dotted center line ("yellow") = PMS 137C
• Track boundaries ("white side lines") = CMYK 0-0-2-0

We were advised that carpet-based tracks have fewer issues with light reflection than their vinyl alternatives.

The spec in Illustrator Format for the track is available to download from the AWS Documentation

We ordered the track in July 2022 and were given a figure of £1314.02 for us to get this printed to carpet using bannerworld.co.uk. It came to us in three sections.

The Barrier

After looking at this A to Z Speedway Printed Wall Barrier for AWS DeepRacer Race Track I felt the cost and issues around delivery to the UK were going to be problematic so I decided to try a build my own.

Spoiler alert, although the barrier worked it is probably the main thing I would change for future events.

The materials used were:

• Green Plastic ‘Corex’ Board. 4mm thick - A1 in Size. I ordered 30 sheets from Vesey Gallery via their EBay shop.

To join the corex board together I used plastic Shower/Bath screen joiner/seals. They came from Shower Seal UK Ltd. I ordered them from their eBay store.

• 8 x 2M lengths of Straight Seals
• 2 x 2M lengths of Corner Seals

I cut these to the length required to fit the A1 corex plastic sheets with a small handsaw.

To provide additional support to the barrier I added some desk draw furniture at regular intervals. The barrier has a couple of issues:

• It's too flexible in the gaps without a desk draw. This meant that the cars could crash into it and cause the corex board to detach from the jointing strips.
• Where there was no gap, the car would occasionally get damaged when it hit the desk draw at high speed. We had to fix the car shells during our event with black gaffa tape. We also had to secure the camera with elastic bands to prevent them from coming loose.

The company logo and DeepRacer event branding was added by ordering large stickers from Sticker Mule 5 copies of our logo printed using the Wall graphics option and sized at 508 mm x 173 mm cost a total of $63.

The Cars

We ordered 2 x DeepRacer EVO’s. I already owned one so we had three cars in total to work with.

Each car comes in two boxes. The original DeepRacer car and then the EVO upgrade kit.

We used Shipito to get the cars shipped to the UK.

For racing events we do not use the EVO Kit. This matches what AWS do at their events.

When racing the cars, we learnt a couple of tips about battery placement. It's much easier to secure the motor battery using the Velcro strip to the top of the compute battery. This allows much faster swapping out of batteries during races.

The cars need to be calibrated following the instructions within the DeepRacer Car console. In practice we found that calibration mode would sometimes not work. When this happens, we learned that the following actions are helpful:

• When working with the cars during calibration place them on something so the wheels don't touch the surface. The masking tape included in the car box works well for this.
• Cycle the power on the car.
• Put the car into manual mode and spin the wheels forward and then backwards multiple times.
• Check the compute battery, if it has two bars or less replace it.

Car Parts

You will need an iPad or similar to control the car speed trackside. We had two iPad minis to ensure we always had one charged up.

We ordered spare batteries for the cars:

• URGENEX 2S Lipo Battery 7.4v Lipo with JST Plug

These batteries come with USB charging cables. We ordered this charger

• Compute Battery

For storage of batteries we ordered a number of these protective bags.

As the cars ship with US plugs we ordered 4 of these Combo Travel Adaptors By SKROSS to allow us to keep up with our battery charging requirements.

A battery tester is also essential to check quickly that the battery is in a good state.

A few of tips on battery health:

• Ensure the compute battery is over two bars full. If it isn't swap it out for a fresh battery.
• Rotate the motor batteries regularly - we wouldn't race a car unless it showed green on the DeepRacer Car console.
• Have a system within the Pit Crew for rotating batteries. We used a large bowl to put used batteries in and someone would pick these up and get them on charge for later use.

Timer

So that lap times can be accurately captured we added a pressure sensor-based timer to the track. By the final race we ended up with two versions of this as a prefabricated module became available.

Timer Option 1

For the first timer I mostly followed the instructions on David Smith's GitHub Repo

For the parts I ordered the following items

• Sound Microphone Sensor
• Thin Film Pressure Sensor
• Electrical Terminal Blocks
• Jumper Wire Set
• Wire Spool Set
• Heat Shrink Pack

After removing the microphone sensor, I soldered a terminal block so the pressure sensor cables could screwed into place.

I soldered two meters of cable to each pressure sensor.

The final connection to the RaspberryPi is shown below. I just used off the shelf jumper cables.

To make the pressure sensor look like a finish line of a race track I bought some thin card, some double sided tape and some chequered flag tape.

The pressure sensors were attached to the card with double sided tape and the chequered flag tape added to the card to create the finish line effect. The wires were run under the carpet track and stuck to the track using green gaffer tape.

Here is a video of the timer being tested.

Timer Option 2

Shortly before our final race the Digital Racing Kings Unofficial DeepRacer Timer became available. We ordered one of these to replace the adapted microphone sensors. We found this much better and required much less calibration.

Schedule of Events

We built up to the final race on the physical track. This took the form of a number of virtual and in person events.

Engagement Sessions

Our AWS account team arranged for David Smith who is a Solutions Architect within AWS ML Thought Leadership to deliver our initial training. David is the person you will often see at the AWS Summits leading the Pit Crew.

David delivered a number of sessions with the CDL team.

The first session covered:

• Intro to DeepRacer, Machine Learning and the reward functions. (26 Mins)
• First Model and Training (36 Mins)
• Q&A (21 Mins)

After the Q&A we had a break and returned a few hours later for further Q&A once everyone had got the chance to build a model.

This session attracted 105 attendees. After the first session David shared a number of follow-on resources to help our team continue their DeepRacer learning.

• DeepRacer Video to learn more about Deep Racer.
• Getting started With DeepRacer
• Train Your First Model
• Input parameters available
• Racing tips
• Deepracer community
• Waypoints
• DeepRacer for cloud
• DeepRacer workshop
• PRO TIP: Suggest setting the hyperparameter of Discount Factor to 0.95
• Workshop console dive
• Workshop pre-recorded session

About a month after the initial engagement session we had a couple of check-in sessions with AWS. The first was a 'pro' session for racers who wanted to do a deep dive on particular topics. The second session was for people who were still getting started with DeepRacer and wanted to ask more basic questions.

About a month before the final race, we had our track in place and a timer working so we started to offer weekly check-in sessions trackside. These sessions allowed anyone to bring their model along and a member of our Pit Crew would help the team race. We also offered a remote service where people could send a message with their model, and we would record the car going round the track.

This time on track proved one of the most popular sessions and we were often oversubscribed. Having the chance to tinker with the track, cars, play about, crash the cars etc also helped train our Pit Crew for the final race.

Other Engagement Tips

In between the engagement sessions we tried a number of things to keep people interested and build the momentum towards the final race.

• Blog Posts: One of the Pit Crew wrote about their experience of training models and offered hints and tips.

• Race Chat: We used MS Teams to host a chat with the racers. This helped build up a healthy competition between the teams, offer support, answer questions and debug issues.

• Take our models to the in-person London Meetup and test them out.

• Take our models to the AWS London Summit to try them out.

• We also used the chat to post links to the live feeds of the various AWS Deep Racer events that were happening at the summits across the world.

• Virtual League. Within the DeepRacer section of the AWS console we created a CDL league. This allowed our racers to submit their models to the virtual league over the three months build up to the final race.

The Final

The Pit Crew

The Pit Crew are the hardest working people during the DeepRacer event. Do not underestimate how vital they are to keep the event running smoothly. Our event had 26 teams / 72 people taking part.

• Track Boss: Monitoring the car as it races on the track and putting it back on the track if it crashes. This is hard work. Rotate the person after each race.
• Mechanics: Charging batteries and swapping them out within the cars and doing the calibration. Fixing the cars if any parts fail.
• Lap Timer: Working with the track boss to capture complete laps and disqualify any invalid laps.
• Team Liaison: Finding the next team to race, explaining how to use the DeepRacer Car console via the iPad and getting them ready to race.
• Commentary: Helping the event flow for those watching in person or via live stream by making announcements, doing interviews or general comments on the action.
• Merchandise: Dealing with the free SWAG
• Audio Visual: Connecting AV equipment, sound checking and checking on the live stream.
• Official Photographer: Capturing the joy / pain of the winners / losers.

Racing Structure

Our Pit Crew also established the running order for the event. Each race is three minutes. Within the three minutes each team can complete as many laps as they are able to. The fastest of their laps is what establishes their position on the leader board.

As we knew the number of teams taking part, we did some estimates of how long it would take to put the car on the track, get each team in position and complete three minutes of racing. We validated with AWS that 5-7 minutes per team was about right. This did work out correctly for us but the Pit Crew didn't get any time to relax. We're told this is normal for the AWS run event too and there is an unwritten rule that everyone on the Pit Crew should be active at all times.

We held two racing sessions with an hour break for lunch. Starting at 10AM we got all teams to complete two three-minute races by around 3:30PM.

There was some debate amongst teams on if they should try two different models. Whilst we didn't come up with any real conclusions on this point, we did notice the following real-world effects.

• Environmental factors can impact model performance. We did some prior testing with lighting for example and went with what we found to be most reliable.
• Healthy competition amongst teams led to a feeling that certain cars were more reliable than others.
• We tried to support teams by advising them to start their models at a slower speed and experiment with the speed adjustment via the iPad over the three minutes.
• Once a speed benchmark is established further adjustments can be made around corners and straight sections of the track.

Event Space

In addition to the track, we added screens so people could keep an eye on our leader board and watch the racing. We had two cameras on the track and connected these back to a Blackmagic ATEM Mini Extreme. This allowed us to record all camera and graphics feeds for editing after the event and also live stream via MS Teams to the whole company.

In preparation for our event I attended a couple of DeepRacer events at AWS Summits and also through the UK DeepRacer Meetup. The latter was hosted by JP Morgan who have been doing events for a long time. Having in person conversations with the community members, JP Morgan and AWS was really helpful to me.

DeepRacer Event Manager (DREM)

DREM is an AWS built application that is the missing piece for any DeepRacer event. If you have ever raced at an AWS Summit this is the system in use at those events.

At the time of writing, it is working its way towards a public beta but we were fortunate enough to get our hands on it via our AWS account team. DREM made the event run much smoother and has some awesome features just not available via the public AWS or Open-Source tools that we found.

• Allows racers to upload their models via login to their DREM account.
• Car management, which includes loading models to the cars. Effectively you have a fleet of cars registered in DREM and can send the model to any car. Doing this manually via separate logins to each car console now feels painful after using DREM.
• Integrates with the lap timer device recording all key lap data.
• Uses lap timer data to create a live scoreboard for the event.
• Provides on-screen graphics showing the current racer and their lap time stats such as current and best.
• Provides a console for the Track Boss / Pit Crew to invalidate laps e.g. when the car leaves the track and crosses the pressure sensor incorrectly.

I can't wait for the beta version to be available and recommend that anyone doing a DeepRacer event speaks to AWS about availability of DREM.

Prizes

We offered a prize to the team for the fastest lap and another prize for a 'highly commended' category. The latter was intended to be a way for us to incentivise people to take part in the final race even if they had not done that well in the virtual league.

In practice we found the virtual race to be very different to the physical race. There are far more variables in real life and the virtual race is more of a dust-free lab.

Swag

As an added incentive to take part in the racing we arranged a selection of DeepRacer swag. In the morning of the event, we only allowed our racers to pick up swag. In the afternoon we opened up the swag to everyone.

To capture feedback on the event we made swag pickup conditional on completion of a survey. See results later on.

The T-Shirts were kindly arranged and provided by our AWS Account team. They also designed the logo for the event.

The rest of the Swag came through contacts in the DeepRacer Community.

This slack workspace is where the DeepRacer community get together and talk about all things DeepRacer. The community was an invaluable resource for ideas and support during the run up to our event. A special thanks to Tomasz Ptak who put me in contact with Promo Veritas as they sent us DeepRacer Hoodies, Caps and Socks.

Costs

Overall, we had capital costs of £4,973.22

• The two DeepRacer Cars cost £1,507.48
• Various cables and dedicated 4G router cost £474.59
• DeepRacer AWS charges not covered by our APN credits £623.66
• The barrier £799.47
• Track and lap timer £1,415.79
• Prizes £152.23

We estimated that our AWS bill for DeepRacer would come to $3719. This was covered by Innovation Sandbox Credits. The AWS innovation Sandbox Credits are designed to help you effectively integrate AWS services into your solution or launch a product to general availability on AWS. The AWS Innovation Sandbox credits help offset AWS usage costs incurred during the development. This benefit is available to AWS partners that build or offer services and solutions.

Results

We had 71 people respond to our survey. 100% of the number signed up to race. After the event we also held a retrospective to help us form suggestions for the future of DeepRacer at CDL.

94% said they would take part in another CDL track event with 90% wanting to take part again in the virtual aspect of DeepRacer.

All respondents thought that DeepRacer encouraged or improved collaboration within our teams. 96% of people thought the ~£5k expenditure was good value for money.

87.4% of people reported learning more about Machine Learning by taking part.

We also captured free text feedback:

"Great event, would like to see this on an annual basis."

"Found training on the physical track invaluable. There are quite a few differences in behaviour between the virtual and physical track."

"To properly explore the different factors that go into a successful model takes time. Would happily have continued exploring different models and different approaches to the models with more hours of training."

"Small team, so was easy to engage with each other and bounce ideas off each other."

"All in all, a very enjoyable way to get into AI and machine learning."

"Great use of space. Fantastic collaboration opportunity. Great team building. Fun."

"Cracking thing to do. Encourages a very social twist on a tech thing, it improves diversification with gamified learning which a lot of people benefit from."

What's Next?

We are currently looking at running future DeepRacer events. We are particularly interested in running an event with one or more of our customers. We are also looking at contributing to the community initially through a DeepRacer event for a local college or university.