Final year project selection is stressful enough on its own.
Now imagine two friends sitting in a class, one laptop between them, trying to pick a single project that would satisfy a supervisor, an examination committee, and two completely different career paths.
That was me and Hassan in early 2024.
I was deep into AWS - ECS, Lambda, Terraform, the whole cloud stack. Hassan was into Unity and game development. We needed something that genuinely needed both of us. A game with a cloud backend made sense - Hassan owns the game, I own the infrastructure, we both have something real to show.
We went to the committee feeling good about it.
Then they said four words that changed everything.
"You Need to Add AI."
Not a suggestion. A requirement.
We walked out of that room and looked at each other. Hassan said: "Let's just add some pathfinding and call it AI."
I wasn't satisfied with that. If we were going to add AI, I wanted it to actually mean something - something the committee hadn't seen before, something that justified the domain split between a cloud guy and a game dev.
Hassan had been reading about Reinforcement Learning. The idea was compelling - instead of scripting NPC behavior manually, let the agent learn through trial and error. Reward good behavior, penalize bad, and over time it develops its own strategy.
He pitched it. It sounded perfect on paper.
Then we actually tried to implement it.
Why Pure RL Was a Problem
RL has a dirty secret that papers don't emphasize enough: it needs thousands of training episodes before it does anything useful.
In a controlled grid environment with a simple reward function, that's manageable. In a real Unity game with a 3D environment, dynamic player behavior, and a demo deadline four months away - it's a disaster.
The problems we hit:
RL is unpredictable by nature. That's literally the point - it explores. But in a game, "exploration" means your NPC stands in a corner, walks into walls, or runs away from the player for no reason. Not exactly the threatening enemy we were going for.
Reward shaping is an art, not a science. If you reward the NPC for getting close to the player, it learns to get close - then just stands there. Every reward signal has to be carefully designed or the agent learns to game the reward function instead of actually playing.
Training time. We didn't have a GPU cluster. We had Hassan's laptop. Full RL training on a live Unity environment wasn't going to be ready for demo day.
Hassan was honest about it - the RL implementation was beyond what he could execute in the Unity codebase within the timeline. He built the FSM instead, which worked, but felt incomplete given what the committee was expecting.
That's when I stepped in.
The Research Rabbit Hole
I took ownership of the AI architecture problem. If RL alone was too unstable and FSM alone was too rigid, there had to be a middle ground.
I spent two weeks reading - research papers, game AI blogs, anything I could find. The pattern that kept appearing in serious game AI literature was a hybrid approach. Not RL or FSM. RL and FSM together.
The insight was simple once I found it:
FSM and RL have exactly opposite weaknesses.
FSM is stable but rigid - it does exactly what you tell it, nothing more. Players figure it out fast. The NPC becomes a puzzle instead of a threat.
RL is adaptive but unpredictable - it can surprise you, but early in training it's chaotic and unreliable. You can't ship a game where the enemy randomly walks into walls.
What if FSM handled the high-level structure and RL handled the low-level execution?
The Architecture We Landed On
The FSM controls the overall zombie behavior. High-level states - Patrol, Stalk, Engage, Retreat - with deterministic rule-based transitions. This is what Hassan had already built. It was stable, it worked, it demoed reliably.
But within the Engage state, instead of scripted attack patterns, the NPC uses an RL policy to control its movement.
The zombie knows it's in Engage mode because of FSM rules. But how it moves during engagement - how it positions itself, how it adjusts to the player dodging, how it cuts off escape routes - that comes from the RL model.
The FSM gives the RL agent guardrails. The RL agent never has to explore the entire state space from scratch - it only makes decisions within a constrained, well-defined context. That solved the unpredictability problem.
The RL agent gives the FSM emergent behavior. The high-level states are still deterministic, but the execution inside each state adapts to the specific player. That solved the rigidity problem.
Each one covered the other's weakness. That was the breakthrough.
Where the Cloud Came In
This is where my domain actually mattered.
The RL model needed to be trained on player session data and updated over time. Running that inside Unity wasn't feasible. We needed a backend.
I built a serverless Flask API deployed to AWS Lambda via Zappa, with:
- DynamoDB storing NPC behavior profiles and session data per player
- S3 storing trained model artifacts
- API Gateway exposing endpoints to the Unity client
The split finally made complete sense. Hassan owned the game engine and FSM implementation in Unity. I owned the AI architecture design, the RL research, and the entire AWS backend.
Neither of us could have built the other's half.
The Result
90/100 at the university's Project Exhibition.
Top 8 Runner-Up out of the Final Project Exhibition.
The viva ran long - not because anything went wrong, but because the committee kept asking questions. The hybrid architecture was something they hadn't seen in a student project before. They pushed hard on why we combined FSM and RL, how we handled the training instability, and why Lambda over EC2 for the backend.
We had answers for all of it.
What We'd Do Differently
On the RL side: Train in a headless simulation environment instead of the live game. A simulation running 100x faster than real-time would have produced a significantly better policy by demo day. We were constrained by training in real Unity time which slowed everything down.
On the cloud side: API Gateway WebSockets instead of REST for Unity-to-backend communication. The polling approach added latency that hurt the real-time feel. With my current AWS knowledge I'd redesign that from day one.
On the project itself: Don't wait for the committee to push you. We almost shipped a safe, forgettable project. The "add AI" requirement felt like a burden in that moment. It turned out to be the best thing that happened to us.
The Actual Lesson
The hybrid FSM+RL pattern we built isn't just a game AI trick. The same principle - deterministic rules for structure, learned policies for adaptation - shows up in production systems everywhere. Recommendation engines. Fraud detection. Adaptive difficulty in AAA games.
We didn't invent it. But we found it, understood it deeply enough to implement it under a deadline, and shipped something that surprised the people evaluating it.
That last part - surprising people who've seen hundreds of student projects - is harder than it sounds.
And it started with two friends from completely different domains who had no choice but to figure it out together.
Hassan Saleh Hayat handled Unity, game systems, and FSM implementation. I handled AI architecture research, RL integration design, and the AWS backend.
Source
GitHub:github.com/aboodi679/ai-hitman-backend
Top comments (0)