DEV Community: Andrea

Learning Live Data Systems through an F1 Telemetry Project

Andrea — Mon, 08 Dec 2025 15:53:50 +0000

Overview — From Confusion to Clarity

This project started from a simple desire: I wanted to build something around data collection. At the time, I was deep into observability and monitoring, but somehow I ended up in the world of telemetry — mainly because I was learning a lot about Formula 1 and my interest kept growing every day.

What began as curiosity quickly turned into a challenge. I was a brand-new F1 fan trying to build a system I barely understood. I got confused many times, but I noticed something important: I kept making small milestones, and each one pushed me forward.

Slowly, the project gained momentum. The architecture started becoming clear. Pieces that looked impossible at first suddenly made sense. And by the time the system was running, I realized I had learned more than I expected.

In this blog, I’ll share the core lessons, the architecture decisions, and the parts of the system that were the most fun and surprising to build. So read along on how a simple idea grew into a complete system. The main sections will walk through:

The backend layer in Python
The FastAPI + WebSocket connection that keeps everything flowing in real-time
The frontend side where the data is received, visualized, and even sent back to the simulation.

BACKEND

Designing the Simulation Backend — Where the Real Engineering Began
The backend was really the heart of this whole project, because that’s where the car logic lived. This is the part that actually generated the data I wanted to visualize. And honestly, it didn’t start nicely at all , it began with a few random files where I was just trying ideas, tricks, and anything that came to mind. The architecture looked odd, messy, and unclear.

Every new day I would wake up and try again. Then one day, everything clicked — not magically, but because all the messy files I’d been building finally connected into a clear picture.

I realized the simulation needed to start exactly how a real car starts — from RPM ignition, to engine warm-up, to getting on track. That single insight made the structure fall into place. Before that moment, I was just wandering. After that, the direction became obvious.

The Lesson That Changed Everything: State

The biggest clarity I gained was around state. State is what made the backend truly integrated , it allowed different modules to share data smoothly. Without state, the whole simulation would have collapsed.

Take the engine for example. This was one of the most fun parts to build.

I started with the ECU module (Engine Control Unit). Before this project, I didn’t even know what the ECU really did. Then I learned: it’s the “computer” that controls things like the fuel-air ratio and makes combustion efficient. Cool stuff.

Now, of course, I couldn’t put all combustion logic inside the ECU file. That would create a monster file. So I modularized:

ecu.py handles requests and ratios
fuel.py handles supply
combustion.py handles burn logic

And this is where state became the hero.

The ECU doesn’t produce final fuel burn numbers — it produces a fuel request.

That request is stored in state.

Then the fuel module reads that request, deducts the amount, and updates the state again.

Finally, the combustion module reads the updated state and performs the burn.

It became a smooth pipeline because every module could ask from state and feed into state.

In simple terms:

State became a global dictionary that acted like the car’s shared memory.

Before using state, I was writing extremely long files. My first engine file had almost all logic mashed together. It was unmanageable. Once I introduced modularization + shared state, the backend finally felt like a real system, not a pile of experiments.

This part of the project taught me one of my biggest lessons:

Long simulations only work when every module has memory. Stateless code collapses under complexity.

How self Saved the Simulation (Timing, Sectors, and Real F1 Logic)

After state, the next thing that completely changed the backend architecture was self.

At first it felt confusing, but it became the hidden hero of the whole timing system.

In real F1 broadcasts, you always hear commentators saying things like “Lando purple in sector 3!” or “That’s a green sector for Hamilton.”

So when I was building the timing module, I wanted to simulate exactly that:

Purple → best overall
Green → driver’s personal best
Yellow → anything else

To even do this properly, I had to map the Silverstone track into segments and define sector boundaries.

But here’s where it gets real:

you can only calculate “sector 1 time” after you cross into sector 2.

And this is exactly where self shows its power.

self is what lets the simulation remember things between ticks.

Python runs the simulation in loops (“ticks”), so for example:

In tick 1, the car is in sector 1.
In tick 2, it moves into sector 2.

For the timing file, we must compare:

what sector we were in before → self.previous_sector
what sector we are in now → current_sector

Only self can hold that previous-sector info automatically without relying on some huge manual “previous values” text file or awkward global variables.

Same thing for sector times — we store the timestamp when a sector starts, and when the next sector begins, self is what lets us compute the exact duration of the previous one.

So in short:

state holds global shared values across all modules.
self holds the memory inside the module across ticks.

That combination basically made the timing system feel like a real F1 sector tracker instead of a random counter.

How Everything Runs Together (Classes, __init__, and the Main Loop)

So after understanding state and self, the final piece that made everything make sense was how all these modules actually run together.

Python looks simple, but when you’re combining many modules, classes, and a global state, it becomes wide pretty fast.

To keep it easy, let’s stick with the timing example again.

Every module in the backend was built as a class:

the class has an __init__() → where all the self values get initialized
then it has an update() → which runs every tick and changes those values
and the class takes state so it can read/write shared info

So the idea is:

__init__ = sets up all the memory the module needs (sector numbers, lap counters, flags…)
update = does the logic every tick (detect sector change, update times, apply color logic…)

But none of these modules do anything alone.

Everything comes to life in main.py.

That’s the file where:

all modules (engine, brakes, tires, timing, weather…) are imported
each one is instantiated as a class
the big state dictionary is created
a loop runs all classes’ update() functions in order
and the final processed data gets streamed out through WebSocket

It’s basically like assembling the whole car:

engine module runs
brakes run
aero runs
tires update
timing calculates
weather injects external data
state keeps everyone connected

main.py is the “race director” — it calls each module every tick so everything stays synchronized.

This structure made the backend feel organized, realistic, and easy to extend when new ideas popped up.

FastAPI Server & WebSocket Connection

To stream live telemetry, we needed something more than regular HTTP requests like Axios — we required persistent connections. That’s where FastAPI with WebSockets came in.

We created a server file that defines the FastAPI app and sets up the WebSocket endpoint. This connection is what allows data to flow continuously from our Python simulation to the frontend.

Here, the state object became central. Instead of sending piecemeal data, we just import state and push it as a full object to the frontend. This keeps the architecture neat and makes the flow predictable.

In addition, we added some logic around session duration and modes, like RACE or P1. That’s where the timing control sits — it governs how long the simulation runs for each mode, adding a bit of realism and fun to the simulation.

Connecting to the Backend

On the frontend, to receive live telemetry, we needed a persistent connection to the backend. Using WebSockets, we could stream data continuously, allowing charts, timers, and other UI components to update in real time without reloading or polling.

To simulate the strategist role in F1 teams, we also sent messages back through the same WebSocket. This included commands like pit calls, ERS deployment decisions, and other strategy inputs.

With this bi-directional flow, the simulation felt lively. The frontend could now react to live car data, and at the same time, strategy inputs could influence the backend simulation — covering key aspects of real F1 logic in a simplified but interactive way.

When the UI Started to Feel Like a Race Weekend

This was the stage where the project really came to life. Since everything was running on localhost, I couldn’t just dump all the data in at once — my machine would struggle. So the focus shifted to visualizing live data as it streamed in, similar to what you see on real F1 dashboards.

The first big UI piece was the live charts. After watching a lot of F1 analysis content, I noticed how often wheel-speed and telemetry overlays rely on line charts. That’s why I went with Recharts. I built a reusable chart component that could plot wheel speed or any y axis values over time and also mark sector boundaries. It wasn’t just about seeing the speed — it showed where on the track each spike or drop happened. That little link between sectors and speed made the simulation feel closer to real F1 telemetry.

Then there was the timer counter with the proper F1 coloring logic. I implemented the same color system used in actual broadcasts: purple for overall best, green for personal best, and yellow for the rest. Seeing those colors update live made the sessions feel more authentic.

I also integrated pit logic, so you could simulate in-laps, out-laps, and strategy moments. And on top of that, I added weather data from an API (Meteo), giving the UI small touches like conditions you’d normally see on an F1 dashboard.
Those three—charts, timers, and weather/pit logic—became the main pillars of the frontend

Conclusion
This whole project was really just a leap of faith. I didn’t start with a clear picture or some perfect plan — I just started. And day by day, things slowly made more sense. Every file I wrote, every bug I chased, every small breakthrough added up. Looking back, I’m genuinely thankful to God for that rhythm of trying again each day, because that’s what carried the project from confusion to clarity.

I ended up learning way more than I expected — not just about Python or telemetry or websockets, but about how to think, how to break problems down, and how to keep going when nothing makes sense yet. And those lessons are things I’m still applying even now.
So yeah… what began as a simple idea turned into something that actually works, something I genuinely enjoyed building, and something that built a stronger mindset in the process.

I hope you get the same kind of value from reading this as I got from building it — even if it’s just one idea, one perspective, or one spark to start your own project.

Inside the University Cloud: Solving Real-World Networking Challenges

Andrea — Tue, 02 Dec 2025 15:46:14 +0000

Overview
The university is launching a new Student Portal to give learners a seamless digital experience. Through this portal, students will be able to:

View grades
Access course materials
Pay tuition fees
Submit assignments

To support this rollout, James, the Cloud Engineer, and Sarah, the School’s Solutions Architect, were tasked with building a secure and scalable architecture on AWS — the chosen cloud provider for its reliability and customer-obsessed design.

They began by setting up the core networking environment:

A VPC to isolate all university resources
A public subnet for externally reachable components
A private subnet for backend application servers
An Internet Gateway (IGW) attached to the VPC
A route table to enable internet access for the public subnet
An EC2 instance in the private subnet hosting the student portal backend
NAT Gateway to Allows private servers to reach the internet without being exposed.
Appropriate security groups to regulate inbound and outbound traffic
Network ACLs (NACLs) to provide an additional layer of subnet-level security

Before finalizing the setup, James added a bastion host in the public subnet to allow secure SSH access into the private EC2 instances for administration, debugging, and deployments.

With all foundational components in place, the team prepared to launch the portal and validate that every part of the networking environment worked as intended.

The Architecture

The SetUp

Step 1: Create a New VPC

Log in to the AWS Management Console using your credentials.
Hint: Always create and use an IAM user with the necessary permissions rather than using the root account. This ensures least-privilege access and improves security.
In the search bar at the top, type “VPC” and select Your VPC from the sidebar.
In the VPC Dashboard, you will see a default VPC. For this lab, create a new VPC:
- Click Create VPC.
- Give it a name, e.g., web-portal-VPC.
- Assign an IPv4 CIDR block of 10.0.0.0/16.
- Hint: Use tags to label your resources — this helps organize and identify VPCs later, especially in environments with multiple resources.
Leave the rest of the options as default and click Create VPC.
After creation, you will see your new VPC listed in the dashboard.

Step 2: Create a Public Subnet

On the VPC dashboard sidebar, click Subnets.
Click Create Subnet at the top of the page.
In the creation form:
- Select the VPC you created in Step 1 webportal`-VPC. You will see its IPv4 CIDR block 10.0.0.0/16.

CIDR Calculation & Subnet Planning

- VPC CIDR: 10.0.0.0/16 → 32 is the total bits so it’s ( 32-16) =16 bits fixed, 16 bits available → 2^16 = 65,536 IP addresses.
- Subnet CIDR: 10.0.0.0/24 → 32 is the total bits so it’s ( 32-24)=24 bits fixed, 8 bits available → 2^8 = 256 IP addresses.

How to choose subnet ranges:
- Each subnet’s CIDR must be **within the VPC CIDR**.
- Plan enough IPs for all resources in that subnet (EC2s, NAT, load balancers, etc.).
- Keep future growth in mind — don’t allocate too small a range if you may add more instances later.

Name your subnet: e.g., Public-Subnet.
Select an Availability Zone (AZ):
- AWS regions (e.g., Africa (Cape Town)) contain multiple AZs — isolated data centers.
- Hint: Using AZs improves high availability. If one AZ goes down, workloads can continue in another AZ. Imagine students all over the world hitting the portal at the same time — if only one AZ handled all traffic, it could overload. Using multiple AZs ensures the system can distribute load and remain operational even under high demand.
Assign a CIDR block for the subnet: e.g., 10.0.0.0/24.
- Hint: Subnet CIDR must be within the VPC’s CIDR. /24 gives 256 IP addresses (usable for EC2s, NAT, and other resources).
- This allows you to plan for growth without overlapping other subnets.
Leave the rest of the options as default and click Create subnet.

Step 3: Launch an EC2 Instance in the Public Subnet

Hint / Analogy for EC2 and Instance Setup:

EC2: Think of an EC2 instance as your server in the cloud — it’s where your application actually runs, like a web portal server. This is similar to your laptop, which runs applications locally.
AMI (Amazon Machine Image): The AMI is like the operating system on your laptop (e.g., Windows or Linux) along with some pre-installed software. It sets the base environment so your application can run.
Instance Type: This is like the hardware specification of your laptop — how much CPU, memory, and storage it has. Choosing the right instance type ensures your application performs well under expected load.

Additional Hint for Scaling:

This instance hosts the front-facing portal and all services in our lab. For small labs, one EC2 is enough.
In real-world scenarios, if thousands of students are using the portal simultaneously (e.g., during exams), you can add more EC2 instances or scale up/down dynamically to handle traffic while optimizing costs.

In the AWS Management Console, use the search bar to type EC2 and select it.
On the sidebar, click Instances, then click Launch Instances.
Configure the instance basics:
- Name: Web-Portal
- AMI (Amazon Machine Image): Select an appropriate AMI, e.g., Amazon Linux 2023.
Instance type: Choose a small instance like t3.micro.
Hint: Instance type determines CPU, memory, and network capacity. For a lab web portal, a small instance is enough. In real scenarios, consider traffic load, student activity, and portal features.
Key pair:
- Create a new key pair called webportal.
- Hint: This key allows you to SSH into your EC2 — necessary for administrative tasks, checking logs, or debugging.
Networking:
- Click Edit under Networking.
- Select the VPC you created (Web-Portal VPC).
- Choose the Public Subnet you created earlier.
- Enable Auto-assign Public IP. Hint: Assigning a public IP lets the instance be accessible from the internet
Security Group:
- Create a new security group called WebPortal-SG.
- Hint: Security groups control inbound and outbound traffic. In production, this is dynamic and carefully managed; for this lab, keep it simple.
- Add Inbound Rule:
  - Type: SSH (22)
  - Source: My IP

Hint: This ensures only your machine can SSH into the instance.

Review your configuration and click Launch Instance.
After launch, the instance will appear in the Instances dashboard:
- You can see its Status, Public IPv4, and Private IPv4 addresses.
- Clicking the instance allows you to explore more details, like network interfaces and security groups.

Step 4: Create an Internet Gateway

In the AWS Console, search for VPC and select it.
On the sidebar, click Internet Gateways.
Click Create Internet Gateway.
- Name: webportal-IGW
- Leave other settings as default.
After creation, the IGW is available but not yet attached to the VPC.
Select the IGW and click Attach to VPC, then choose the Lab VPC you created. Hint: Now the public subnet can send and receive traffic from the internet, allowing students to reach the portal.

Step 5: to create the route table:

In the AWS Console, search for VPC and click Route Tables on the sidebar.
Click Create Route Table.
- Name: Public-RT
- VPC: Select webportal VPC
After creation, select the route table, go to the Routes tab, and click Edit routes.
- Add a route:
  - Destination: 0.0.0.0/0 (this means all traffic to the internet)
  - Target: Lab-IGW (the internet gateway you created in Step 4)
Save the route.
Associate the Route Table with the Public Subnet:
Go to the Subnet Associations tab of the route table.
Click Edit subnet associations.
Select the public subnet you created in Step 2 and save.

Hint:

Now your public subnet knows that all internet-bound traffic should go through the IGW.
This completes the path for students to reach the portal: browser → ALB (or EC2) → public subnet → IGW → internet.

The Progress so far

Understanding the Flow: Public Subnet and Internet Gateway

How traffic flows for the student portal:

A student opens a browser and searches for portal.com (our student portal).
The DNS resolves portal.com to an AWS resource. In a production setup, this would be the ALB (Application Load Balancer) DNS name, e.g., aws-1234fe-alb.amazonaws.com.
The ALB receives the request and forwards it to one of the EC2 instances in the public subnet.
- At this stage, the request has already reached the EC2 via the internet and IGW; the subnet’s route table is not involved for incoming traffic.
- Note: A subnet is not public until an Internet Gateway (IGW) is attached and the route table sends traffic to it.
Internet Gateway (IGW) ensures that EC2 in the public subnet can communicate with the internet. Incoming requests from students are received via the IGW.
Outbound traffic / responses:
- When the EC2 instance responds to the student’s browser, the route table associated with the subnet checks the destination (the student’s public IP) and directs the response through the IGW.
- Without the route table pointing 0.0.0.0/0 to the IGW, responses wouldn’t reach the internet.
This completes the round-trip: Student browser → ALB → EC2 → public subnet → IGW → student browser.

Hint:

Route tables primarily control where traffic leaving the subnet goes, and for incoming using the local route
The IGW + route table combination makes a subnet “public”, meaning its instances can respond to internet requests.

Hint:

Public subnet + IGW = internet-facing resources.

Server Side Setup

Kindly note: On the server-side (private subnet) you won’t see UI screenshots here. The setup steps are mostly the same as the public-side resources, with only a few key differences we’ve already highlighted. Keep reading through — everything follows the same pattern.

Step 6: Creating a Private Subnet, NAT Gateway, and EC2 for Backend Services

Now that the student-facing application is running on a public subnet, we need to handle backend services that should not be publicly accessible, such as the grading system or other sensitive components of the student portal. These services must remain private to prevent direct access from the internet.

1. Create a Private Subnet

Go to the VPC dashboard in the AWS console.
On the sidebar, click Subnets → Create Subnet.
Select the VPC you created earlier (Web portal VPC).
Name the subnet Private Subnet.
Choose an Availability Zone (AZ). Hint: Using AZs improves high availability—if one AZ goes down or is overloaded, workloads continue in another AZ. For example, if many students hit the portal simultaneously, multi-AZ ensures traffic is distributed.
Assign a CIDR block that:
- Falls within your VPC range (10.0.0.0/16).
- Does not overlap with the public subnet.(10.0.0.0/23).
- Leaves space for growth (e.g., if you anticipate 254 addresses, choose /25 or /23 depending on needs).

2. Create a NAT Gateway

In the sidebar, click NAT Gateways → Create NAT Gateway.
Select the public subnet for the NAT Gateway. Hint: NAT must live in a public subnet to access the internet.
Assign an Elastic IP (EIP). Hint: EIP ensures a stable public IP so that outbound requests from the private subnet appear consistent to the internet (unlike auto-assigned public IPs that can change).
Click Create NAT Gateway.

3. Route Table for the Private Subnet

Go to Route Tables → Create Route Table → name it Private Subnet RT.
Associate this route table with the Private Subnet.
Add a route:
- Destination: 0.0.0.0/0
- Target: NAT Gateway
- then asscosite it with the private subnet
- Hint: This ensures outbound internet traffic from the private EC2 (e.g., to download updates or call external APIs) passes through the NAT, keeping the EC2 private from inbound internet traffic.

4. Launch an EC2 in the Private Subnet

Go to EC2 → Instances → Launch Instances.
Name it Server-Portal.
Select an AMI (OS) suitable for backend services. Hint: Like before, AMI is the operating system your EC2 will run.
Choose an Instance Type based on expected workload (CPU, memory). Hint: For backend tasks, adjust based on load (grading computations may require more memory/CPU).
Select the Private Subnet for networking.
Assign a security group allowing necessary ports for internal communication (SSH from bastion, application ports from public EC2, etc.).

THE PROBLEM

A few hours after releasing exam results, the university’s IT team started receiving complaints:

“My grades are not appearing on the portal.”
“The page keeps loading forever.”

CloudWatch showed increased errors, but the logs were not detailed enough to reveal the root cause.
So the backend engineering team decided to SSH into the private application server (through the bastion host) to investigate.

THE SOLUTION

So far we have created 2 ec2s try this one alone here are a few instrctions:

Step 7: Create the Bastion Host
The Bastion Host is simply another EC2 instance that acts as a secure jump server to access private resources.

Subnet Placement: Launch it in the public subnet so it can receive SSH connections from your local machine.
Security Group: Create a Bastion-SG that allows inbound SSH (port 22) from your IP only.
- Note: In production, bastion security groups are more restrictive and often integrated with multi-factor authentication or other controls. For the lab, we keep it simple.
EC2 Setup: Choose an AMI and instance type similar to other EC2s. Assign the public IP automatically so you can SSH into it.

DONE🎉

Step 8: SSH into Bastion and Access Private EC2

With the Bastion Host in place, the team could investigate the issue on the private EC2 instance hosting the backend services.

SSH into Bastion Host:
- On Git Bash (Linux terminal), use:
  
  ssh -A -i ec2-user@

This connects your local machine securely to the bastion host in the public subnet.

Prepare Access to Private EC2:
- Go to the private EC2 instance (“Server Portal”) and update its security group to allow inbound SSH from the Bastion-SG.
- This ensures the bastion host can reach the private EC2, while the private EC2 remains inaccessible from the internet.
SSH to Private EC2 via Agent Forwarding:

ssh ec2-user@

This gives secure access without exposing the private key on the bastion host.

Investigate and Solve the Issue:
- Inside the private EC2, the team discovered:
  - The grade-processing service had crashed.
  - A Python script syncing grades had a file permissions error.
  - A critical directory had incorrect Linux permissions after a deployment.
- These issues could not be resolved from CloudWatch or the AWS console — direct server access was required.
Outcome:
- The backend services were fixed, and students experienced fast, reliable portal access again.

ADD ON

Step 9: Creating and Using a Network ACL to Deny Traffic to the Private EC2

1. Go to the NACL page in AWS:

In the AWS console search bar, type “Network ACLs” and select it.
This is where you manage subnet-level traffic controls.

2. Create a new NACL:

Click Create Network ACL.
Give it a meaningful name, e.g., PrivateSubnet-Deny-ACL.
Select the VPC where your private subnet resides.

3. Associate the NACL with the private subnet:

After creating, select the NACL, click Subnet associations, then Edit subnet associations.
Check the private subnet to associate this NACL.

4. Add a deny rule for the EC2 instance:

Select Inbound Rules and click Edit inbound rules.
Add a new rule:
- Rule #: pick the next available number (e.g., 100)
- Type: All Traffic (or specify SSH if you want to limit to port 22)
- Protocol: All (or TCP if limiting)
- Port Range: 0-65535 (or just 22 for SSH)
- Source: the private EC2’s IP in CIDR /32 format (e.g., 10.0.1.181/32)
- Action: Deny

5. (Optional) Outbound rule:

If you also want to stop this EC2 from initiating connections to other hosts, add a similar rule under Outbound Rules.

6. Save changes:

Once done, the NACL is active.
Effect: The bastion host (or any other source) can no longer reach the private EC2, because the NACL explicitly denies traffic.

Hint:

NACLs are stateless, so you must create both inbound and outbound rules if you want to block traffic in both directions.
Rule numbering matters: lower numbers are evaluated first.